⏳
3 mins
read time
The Raspberry Pi is a very popular and versatile device. One can easily plug a camera and start doing some image analysis. I have developed a module to use state of the art object detection models in the captured images and also take photos at regular intervals. In this article I take my project to another level by implementing this image processing to streaming videos.
My project is composed in two components:
The backend api will:
In order to capture the image, I started by using a method from Miguel Grimberg which sends images as a continuous stream of data in a Multipart Response. This method is adapted for cases when there is no image processing to be done or when you have a device more powerful than the Raspberry Pi.
I prefer to reduce the number of send images and send single image image is each http request. This allows me to configure the number of required frames per second in the frontend application and also to turn on and off the object detection module when streaming images.
This can be implemented as a flask route like the following code:
@app.route('/api/single_image')
def single_image():
detection = bool(request.args.get('detection', False))
frame = Camera().get_frame()
if detection:
frame = Camera().prediction(frame)
return json.dumps(dict(img=Camera().img_to_base64(frame),
width=WIDTH,
height=HEIGHT))
The frontend component will fetch the image from the backend route and will render it in the web application. I used the framework Angular to build a robust typescript application. This framework allows me to have an structured developing environment with the components logic and also it has compilation capabilities to compress and optimize the javascript and css code. The resulting html code is served by the backend application.
The html code fragment to render the image is the following:
<img [src]="singleImage.img | imageDecode" width="{{ singleImage.width }}" height="{{ singleImage.height }}">
Note that the imageDecode pipe will transform the image so that it can be read by the browser. The complete frontend code can be found here.
I customized the frontend application using the Angular Material Desing. This allows to have some predefined css styles and also some javascript animations such as hammer.js for gesture recognition.
The frontend application has the following inputs related to object detection:
You can see an utilization example of the application in the following video:
Your browser doesn't support HTML5 video.
A reverse proxy is a component that retrieve resources to a client from other one or more servers. The advantage of using a reverse proxy are:
In addition to the backend application, I also use the celery monitoring tool called Flower. This monitoring tool comes with a Flower API that can be used to retrieve information about celery tasks.
I use the popular nginx webserver as a reverse proxy to serve my backend and Flower application. The configuration of the virtual server can be found below:
# /etc/nginx/conf.d/backend.conf
upstream backend {
server 127.0.0.1:5000;
}
upstream flower {
server 127.0.0.1:5555;
}
server {
listen 80;
listen [::]:80;
server_name raspicam raspicam.local 192.168.0.1;
gzip on;
gzip_types text/plain application/xml image/jpeg;
gzip_proxied no-cache no-store private expired auth;
gzip_min_length 1000;
location /flower/ {
rewrite ^/flower/(.*)$ /$1 break;
proxy_pass http://flower;
proxy_set_header Host $host;
}
location / {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
Flower request have a /flower/ prefix and will be redirected to the flower
application running at the port 5555, the rest of the request will be directed
to the backend application.
This application shows how to combine resources from a Raspberry pi with object detection algorithms in video streaming application.
The most popular use case for this application is a surveillance system, where the device regularly analyse the image and also which can be monitored in more details by streaming the video.
In the near future I plan to add more analysis capabilities to the frontend application so that one can easily explore the metadata of the collected images.
I have been working in this project since some months as you can see in the related articles. My application code is published in github and it’s my most popular repository with more than 15 ⭐. You can add your own if you like.
This project implements a deep learning model on a Jetson Nano to count and track people passing in front of a video camera.
The photo frame turns on when it detects movement and shows nice pictures. Photos are shared with a Samba server.
In this article I show how to use a Raspberry Pi with motion detection algorithms and schedule task to detect objects using SSD Mobilenet and Yolo models.
This post how how to implement a light object detection algorithm