⏳
7 mins
read time
Raspberry Pi are small devices that can be combined with captors to get information from the environment such as cameras, microphones or temperature sensors. In addition they have a fair amount of computational power in order to be used for edge computing.
In this article I explore some applications using a PiCamera and computer vision library OpenCV.
Deep learning algorithms are the first AI application that can be used for image analysis. Some of the most poplars algorithms that can be used in Raspberry Pi environments are SSD Mobilenet and YoloV3 since they are light and have a good quality/price ratio.
In a previous article I compared these two algorithms using the deep learning module from OpenCV.
Since then I have made some progress to my implementations by cleaning the code and using more numpy matrix operations instead of for loops and especially thanks to an optimized version of OpenCV for the Raspberry Pi.
The traditional version of OpenCV proposed by Raspbian repositories comes from version 3.3 which is late compared to the most recent one and it’s compiled using the native compiler for ARM architectures. The version proposed by Dario Limongi is compiled using TBB, Neon and VFPV3 libraries, which raises the potential of computer vision applications for the Raspberry Pi.
Adrian Rosebrock, from PyImageSearch, states that this optimized version gives 30% global speed increase and 48% speed for DNN OpenCV module.
I have installed this Faster OpenCV version and run the same test for detection using SSD Mobilenet and Yolo detection. The results are showed in the following table:
| Device | SSD Mobilenet | Yolo V3 tiny | ||
|---|---|---|---|---|
| Model loading [s] | Inference [s] | Model loading [s] | Inference [s] | |
| PC | 0.5 | 0.19 | 0.091 | 0.2 |
| Raspberry | 2.97 | 0.6 |
I’m pretty happy with the results. The gain in speed is approximately of 44%. SSD Mobilenet is still the fastest from both of them and I prefer it’s detection performance.
I must say that installing this OpenCV version is also very easy since we can find the compiled .deb files. The complete instructions can be found at his github page
Another popular application for Raspberry Pi is home surveillance. This can be achieved using motion detection algorithms. A simple implementation can be done by:
This simple algorithm can be used to spot the difference for two pictures. I used my motion detection algorithm for the following Sponge Bob pictures in order to find the differences between the two pictures. It can detect the 3 differences from Sponge Bob and also the difference on the image borders (that I made when cropping the original image to obtain image 1 and 2).
Image 1 from Sponge Bob
Image 2 from Sponge Bob
Differences from Image 1 and 2 of Sponge Bob
I organized my code in a python class that contains 3 methods:
image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
image = cv2.GaussianBlur(image, (21, 21), 0)
if self.avg is None:
self.avg = image.copy().astype(float)
cv2.accumulateWeighted(image, self.avg, 0.5)
frameDelta = cv2.absdiff(image, cv2.convertScaleAbs(self.avg))
thresh = cv2.threshold(
frameDelta, DELTA_THRESH, 255,
cv2.THRESH_BINARY)[1]
thresh = cv2.dilate(thresh, None, iterations=2)
cnts = cv2.findContours(
thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
Soft image 1
Average image 1
Delta between two images
This algorithm has very low power consumption and during my tests it was 10 times faster than SSD Mobilenet. The only disadvantage is the fact that it needs an almost static background to work well, if not it will increase false positive errors. Still it’s very light and can run near real time in a Raspberry Pi as you can see in the following video:
Your browser doesn't support HTML5 video.
Running these algorithms in the Raspberry Pi increase the temperature easily and if you don’t have the correct cooling it can damage your device. This is why I prefer to run this task at regular intervals.
There are several ways to do this:
time.sleep() functionI prefer using a task manager in order to have a more detailed control of the tasks. I decided to use Celery, which is an asynchronous task runner, it allows you to turn your function into a task that will be executed in the background. It’s specially adapted for long task and it can be distributed in different workers. It will nicely handle all problems with your script, it can retry a task, report problems log, etc.
Turning the computer vision function into a Celery task is easy, I just need to create a Celery instance and decorate the task with Celery task decorator. The regular task can be added to the Celery instance in thebeat_schedule parameter.
celery = Celery("app")
celery.conf.update(
broker_url='redis://localhost:6379/0',
result_backend='redis://localhost:6379/0',
beat_schedule={
"photos_SO": {
"task": "backend.camera_pi.CaptureContinous",
"schedule": timedelta(seconds=int(str(os.environ['BEAT_INTERVAL']))),
"args": []
}
}
)
@celery.task(bind=True)
def CaptureContinous(self):
with PiCamera() as camera:
# rest of the code
In order to call the Celery instance, I can use the following command:
python3 -m celery -A backend.camera_pi worker -B --loglevel=INFO;
-A parameter stands for the app to be controlled-B parameter stands for beat in order to launch recurrent task--loglevel is self explanatory.Celery needs a database to manage tasks. Some of the most popular are redis and Rabbitmq.
You should see in logs that Celery is up and running, scheduling task at regular intervals:
[2019-08-11 14:35:38,480: INFO/Beat] beat: Starting...
[2019-08-11 14:35:38,579: INFO/Beat] Scheduler: Sending due task photos_SO (backend.camera_pi.CaptureContinous)
[2019-08-11 14:35:39,030: INFO/MainProcess] Connected to redis://localhost:6379/0
[2019-08-11 14:35:39,080: INFO/MainProcess] mingle: searching for neighbors
[2019-08-11 14:35:40,186: INFO/MainProcess] mingle: all alone
[2019-08-11 14:35:40,237: INFO/MainProcess] celery@raspicam ready.
[2019-08-11 14:35:40,368: INFO/MainProcess] Received task: backend.camera_pi.CaptureContinous[555ecac8-122a-4a87-8973-c3609de556aa]
[2019-08-11 14:35:43,626: INFO/ForkPoolWorker-2] Task backend.camera_pi.CaptureContinous[555ecac8-122a-4a87-8973-c3609de556aa] succeeded in 2.253857854026137s: None
Celery comes with different tools to monitor tasks:
celery -A proj statusOne can monitor tasks that runs on a remote client by specifying it’s own broker IP address and port:
celery flower --broker=redis://192.168.43.1:6379/0
I’m satisfied with the results. It’s has been almost 5 months since I started to take some photos regularly using the SSD Mobilenet model to filter only photos that contain an object. The Raspberry Pi has been able to do that and also some other tasks. You can see some examples of the photos that I have been able to take during the days:
Tree evolution during spring transition
Rapsberry Pi are very versatile units, nowadays there are some devices that are are specialized to run deep learning algorithms such as coral dev board or the jetson nano from NVIDIA. However Raspberry Pi stands as the most popular device by far and has a great community behind which makes easy when building some projects with.
Running long and recurrent task using a task manager such as Celery has the advantage of running your process in the background, so that any other tasks that runs in the same device are not directly affected. For example I also have a server app to stream video from the Pi Camera, so having Celery simplifies my code, which I will talk in the future.
As always the code is available in github.
This post how how to implement a light object detection algorithm
FPGA implementation of a highly efficient real-time machine learning driving assistance application using a camera circuit.
This project implements a deep learning model on a Jetson Nano to count and track people passing in front of a video camera.
This post shows how to combine resources from a Raspberry pi with object detection algorithms in video streaming application.