Internet – N4N.US – Nerd 4 Nerd

MQTT – Finally!

After a few months of off and on again trying to get my data parse, collect and displayed the way I wanted it. I finally found a little tid bit of info that pointed me in the right direction.

rtl_433 -F “mqtt://192.168.1.15:1883,retain=0,devices=Pipedream/TempSensors/[protocol]/[id]” -M utc

As you can see I am using rtl_433 to receive data from various sensors around the buildings. It gets sent to our MQTT Broker for redistribution to computers that have subscribed to selected topics. Since we have a few sensors of the same model we use [protocol] to group those sensor together, but each sensor has its own feed.

Funny thing is we are now finding sensors, we didn’t know about. We will be watching the data being collected and see if we can figure it out.

Below are the sensors we need to figure out (what they are) 211 and 17. All the rest (so far) we can already id and parse the data correctly.

Finally Posting Data from 433mhz

Would not believe the hoops I had jumped thru to get the data where I wanted it. First 433mhz posts to MQTT, then I have a web page running in a loop, on an old Xserve that reads the data from MQTT. After reading data, it parses the data into CSV files, one for each device. Another web page then uses CSV data to create data blocks.

Could not use CSV directly from MQTT because it would just stop working after the file got so big (32mb I think).. JSON has the same issue, but a bit larger 64mb.

Data is not 100% real time, but it is only, at most 2 minutes or so old for each sensor. Each sensors sends data at its own rate. The receiver, caches and process the data, in batches every 10 seconds. Guess if data comes in during that time it is possible it could get lost, but I don’t think so.

Enjoyed working on this. Really hope to add more sensors and make the data look more sci-fi’ish. But that is for a later date.

UPDATE: In this post, we were using FEEDS by Weaver’s Space, in RapidWeaver. Turned out not to be the best solution for the problem

MQTT – Finally

Finally able to Subscribe and Post Data to a MQTT broker/server.

As always, examples found on the internet, are incomplete, or are just copies of someone else’s post. One of the most popular examples has both Post and Subscribe in one, but of course doesn’t really work without a great deal of review.

Basically you have a server, what they call a broker, that receives all the MQTT traffic., then it allows others to subscribe and monitor the data being posted.

So my setup has a rtl_433 receiver that collects all the sensor data from 433 devices and parses it. Sensors include weather stations, remote temperature, door switches and motion detectors. At present all I am handling is the temperature sensors. Each sensor has an ID which I track, noting it location on the property. Then they also have temperature (of course) and humidity. The sensors sends a lot of other data, but that is all I really need. I then send the data back to broken parsed, into another channel for my use..

The coal is to parse the incoming data into a control panel that will display sensor data, like a dashboard. Basement sensor always in same location, for example. Graphically.

Sensor appear to just send data in a timed manner, once every few minutes. They don’t care if anyone receives it, just throw out over the airwaves. The SDR receives the data, parses it, the Posts to the Broker. The Broken in turn basically doesn’t the same thing as the sensor, it just rebroadcasts the data for anyone that has Subscribe to that Topic. My webpage Subscribes to the Topic and reparses the data and displays it.

Getting there!

DNS Blocking

Had a user bring us a Windows 10 HOME computer that wasn’t able to get on the internet using Wifi or Ethernet. After toying around with it for some time here is what we notice.

Would receive and IP address from DHCP server, but no Internet access.
Could add a new port, usb ethernet, gets IP but still no Internet.
Went through Google search finds to reset network, still nothing.
Removed all the ports, drivers etc.. Still, nada

We found it odd that the computer would get an IP number from server, but then not do anything else with network. We found the user had install AVG anti-virus, took some work to remove without internet, but finally won over. However even after repeating all the steps above, no Internet. Checked resources and system apps, couldn’t see anything that stood out..

We figured it was a typical virus that ransomware the system, but none of the normal flags popped up..

So we finally just reinstalled the OS, oh we did try recovery, that failed. After the reinstall of course everything worked fine. A bit more digging and we found out this ‘issue’ is called Gateway or DNS blocking. Nasty beast. Anyclues how to get around it without rebuilding the whole system?

The Cloud – Really!

Just thinking:

After watching the development of “The Cloud” as a marketing tool for software companies, I’ve decided it is a con, a way for the developer to control the user and milk them for more money.

There are a many ways to utilize “The Cloud” the two major ones are as a personal storage space like DropBox, OneDrive (and its many incarnations) and iCloud, and then there is the way Nest and many many Phone apps developers store your data on their servers for ‘security reasons’. Which is just a nice way of saying, oh yeah, we will charge you an unnecessary amount each month to use their app. After you have a few of these apps, the monthly charges really add up.

What happened to Individual Cloud storage, putting your own NAS on the internet and using that? (Hint: Most users don’t hold the skills to setup, no offense), or their ISP makes it almost impossible.. So everyone has to pay more for less.

[:en]CloudAtCost – VPN[:]

[:en]After a friend that works with IBM suggested I try vpn, and a company called CloudAtCost, I figured I’d give it a try.

Ended up buying three packages with CloudAtCost. One cost less than $13, 1 node, 2 ip numbers, 1 cpu and 512 megs of ram. So figured for the cost I’d try a bigger option, but made a mistake when purchasing, and ended up with 2 developer packages, both at less than $28..

configured one node to be a vpn server. Using OpenVPN was able to create a vpn server and client, that worked pretty well. So tried to manage many web site clients though this system.

Soon i couldn’t access any of our clients sites from my home where I work remotely. Took two weeks to resolve the issue.

Our ISP blacklisted my home static IP.

After removing all VPN software, getting a new router and modem, sending at least 3 tickets to ISP over a weeks time. They finally replied, oh, you were blacklisted.. I wasn’t happy, but wasn’t too mad, since the problem was resolved.. And it wasn’t on our network (my job), so. New equipment that was getting old was replaced. Just wish ISP would have informed/resolve their issue faster.

UPDATE: After a couple months of using CloudAtCost, we ended up having a password issue. We requested help, many times, never received any answers! .. So basically the system is useless. Do not recommend this service!

Outdoor Sensor Array

How many sensors can we plug into one Arduino and setup outdoors? This is how many I was able to cram into one Project Box.

The system uses a ESP8266 Wifi / Arduino Card with a screw terminal Shield. — I attempted to run it all with two 12 volt solar panels with a solar charger. Didn’t have enough juice, so ended up just plugging into outlet located outside.

Hardware