Category Archives: humidity monitoring

Data Logging with the Raspberry Pi pt. 2

Posted on by

Background

In the first article I described the basic system that recorded temperature and humidity, saved it to a local file and also sent the file to dropbox. This simple system worked well but had some limitations:

  • the data file will grow very large and repeated sending to Dropbox becomes very slow and potentially less reliable.
  • data is only available as a comma delimited file which has to be imported to some other app to be useful
  • the raspberry pi zero, whilst an excellent little device, does not lend itself to being a web server that is easily and securely accessed from outside of the local area network (LAN)

In this article I address these issues and add some bells and whistles, e.g. optional notifications by sms and an optional relay controlled heater interface .

Requirements:

  • A web server somewhere that you can use to run php scripts. It does not have to be  your own domain. There are many free offerings that allow use as sub-domains for free, e.g. https://www.000webhost.com/ (not tested and just a suggestion!)
  • Internet connection that allows access to the web server to edit the php scripts
  • For all the functions to work, a Dropbox account
  • Email address(es) if you want to receive notifications
  • The raspberry pi hardware as described in part 1.
  • An account with an sms provider if you want sms alarms sent e.g. smsbroadcast.co.uk. We use this provider and have found them to be relaible, and relatively cheap.
  • Optionally a relay interace for the pi ifyou want to control a heater

No modifications to the original hardware are necessary but SSH or SFTP access to the pi using something like Putty or FileZilla is necessary. We use Putty on a Windows PC with the built in Nano editor on the pi. This article also assumes that you have successfully completed the steps in part1.

Step 1

Modify the python code in file pithlog.py on the pi as follows:

#!/usr/bin/python3
#import libraries
import Adafruit_DHT,datetime,dropbox,urllib.request,urllib.parse
import RPi.GPIO as GPIO
import ssl,os
GPIO.setwarnings(False)

#set up I/O
sensor=Adafruit_DHT.DHT22
#relay control pin 27
relay=27
GPIO.setmode(GPIO.BCM)
GPIO.setup(relay,GPIO.OUT)
#set the pin for DHT22
gpio=17

#start reading
humidity, temperature=Adafruit_DHT.read_retry(sensor,gpio)

#we have a heater relay set to switch off at 20C, on at 18C
if temperature > 20:
GPIO.output(relay,GPIO.LOW)
if temperature < 18:
GPIO.output(relay,GPIO.HIGH)

#this block is optional and can be commented out if no sms alarms
#if the temperature is extremely high (>40C here), send an sms alarm message
#note that this will happen at EVERY reading interval for now
if temperature > 40 :
url=”https://api.smsbroadcast.co.uk/api-adv.php?username=YOURUSERNAME&password=YOURPASSWORD?&to=YOURPHONENUMBER&from=YOURSOURCE&message=YOURMESSAGE%20temperature%20alert%20>=40C&ref=00003&maxsplit=5&delay=0″
requ=urllib.request.Request(url)
response=urllib.request.urlopen(requ)
responseData=response.read()
#print (responseData)

#log the reading
logfile=”/home/pi/tdata1.csv”

#get the time of the reading
timestamp=datetime.datetime.now()

#if the file is over 500000 bytes, start a new one
newfile=str(timestamp.strftime(“%Y%m%d%H%Mtdata.csv”))
timestamp=str(timestamp.strftime(“%Y,%m,%d,%H,%M”))
filelen=os.path.getsize(logfile)
if (filelen>500000):
os.rename(logfile,newfile)

#optionally send the data to a server
#comment out this block if not used
url=”http://www.YOURDOMAINHERE.co.uk/mondata.php”
temp=str(“{0:0.1f}”.format(temperature))
hum=str(“{0:0.1f}”.format(humidity))
tdata=str(timestamp+”,”+temp+”,”+hum+”\n”)
#print (tdata)
readings={‘tstamp’ : timestamp, ‘temperature’ : temp, ‘humidity’ : hum}
data=urllib.parse.urlencode(readings)
data=data.encode(‘utf-8’)
requ=urllib.request.Request(url,data)
response=urllib.request.urlopen(requ)
responseData=response.read()
#print (responseData)

#save the data locally in a file
df=open(logfile,”a”)
df.write(tdata+”\n”)
df.close()

The above code will run according to your cron schedule as set up in part 1. Note that the dropbox backup is now added separately once per day as detailed below. All the files in /home/pi are saved.

Step 2

Install Dropbox-Uploader from Git by following the very clear “Getting Started” instructions in the README.MD file.
Using your editor creat a file called backup.sh and add the following:

#!/usr/bin/env bash
#
# backup files (and only files, not folders) in /home/pi to dropbox
# designed to be run from cron or when needed every now and again
# skips files already present and unchanged
#
# Uses Dropbox-Uploader by Andrea Fabrizi avaialable on Git
#
# backup pi home files
/home/pi/Dropbox-Uploader/dropbox_uploader.sh upload /home/pi/*.* /
#

Step 3

Now edit your crontab file (crontab -e) to add the line:

57  1  *  *  * /home/pi/backup.sh
If you are uploading the readings to a web server, you must now add a script called “mondata.php” in the root folder of the web server:
<?php
//uncomment following line if you want to see contents of POST when debugging
//print “$_POST[tstamp] $_POST[temperature] $_POST[humidity]”;
$dline=$_POST[tstamp].”,”.$_POST[‘temperature’].”,”. $_POST[‘humidity’].”\n”;
$dfile=fopen(“/home/johno/data/tdata.csv”,”a”);
fwrite($dfile,$dline);
fclose($dfile);
//print “OK”;
?>

This will create a csv file “tdata.csv” that will record each reading set as it is taken.

We now have a system that will allow us to control a relay according to temperature, optionally send alarm messages via sms as well as potentially being ready to display the data remotely. Th enext post will show how to display the logged data in a convenient graphical format.