For the last 15 years, the IT industry has tried to simplify managing workloads by bringing them into huge data centers. Cloud computing has allowed companies to scale on demand, lower costs, and (often) externalize operations.
Today, more and more devices are connected and creating more and more data. Smarthome devices, sensors, and cars have mobile connectivity to track consumption, driving habits, location, and more. All this data gets sent back to these centralized data centers, where artificial intelligence (AI) and machine learning (ML) models allow you to process that data and get preliminary insights before sending it to the cloud.
However, the more data you generate, the more expensive it is to transfer it to the cloud. This proliferation of data means it's increasingly important to enable treatment and processing close to where it gets generated.
So what's the goal? You want to run AI/ML algorithms on very small devices worldwide without creating problems. What if you could use the same mechanisms you use in the cloud to run these workloads? This would enable a developer to write an application and run it whether it gets deployed in a cloud environment or an edge computing device.
This is why we created MicroShift, which may help IT systems architects who are looking to manage cloud-native and edge workloads at scale.
Edge devices deployed in the field pose very different operational, environmental, and business challenges from cloud computing. These motivate different engineering tradeoffs for Kubernetes at the far edge than for cloud or near-edge scenarios. MicroShift's design goals cater to this:
- Make frugal use of system resources (CPU, memory, network, and storage)
- Tolerate severe networking constraints
- Update (especially rollback) securely, safely, speedily, and seamlessly (without disrupting workloads)
- Build on and integrate cleanly with edge-optimized operating systems like Fedora IoT and RHEL for Edge
- Provide a consistent development and management experience with standard OpenShift
[ Learn how to accelerate machine learning operations (MLOps) with Red Hat OpenShift. ]
Demo: AI/ML at the edge
This demo aims to run a face detection and face recognition AI model in a cloud-native fashion using MicroShift in an edge computing scenario. To do this, we used the NVIDIA Jetson family boards (tested on Jetson TX2 and Jetson Xavier NX).
In the diagram above, you can see the basic components of this face recognition use case. There is a lot of documentation about deploying MicroShift, so we'll assume you have it running on your NVIDIA Jetson board.
Two different pods run as deployments. One pod is a WiFi access point that exposes a predefined SSID wireless network. This network gets used by the ESP32 cameras to send their video stream to the Jetson board. The other pod is a straightforward web server that receives registration requests from the cameras. Once a camera is registered, it starts a new video-streaming thread to process a certain amount of frames. We based the face recognition library on Dlib, and we use OpenCV to (for example) draw the squares on top of the faces. We want to show two different algorithms (one using just pure CPU and the other using the embedded GPU from the Jetson board) and see resource consumption.
All the code is open source, and you can find it on Red Hat's Emerging Technologies GitHub.
This demo uses ESP32CAM boards, which are very cheap Arduino-friendly cameras that you can easily program.
The code was built using PlatformIO with the Arduino framework. VSCode IDE has a very convenient plugin that allows you to choose the platform you use and compile the code.
Many tutorials exist for using an ESP32 camera with PlatformIO. One of the most relevant files to look at is
platformio.ini, located in the
[env:esp32cam] #upload_speed = 460800 upload_speed = 921600 platform = espressif32 board = esp32cam framework = arduino lib_deps = email@example.com monitor_port=/dev/ttyUSB0 monitor_speed=115200 monitor_rst=1 monitor_dtr=0
Once you connect the camera to your laptop, a new
/dev/ttyUSB0 appears, and you can flash or debug new firmware into the cameras using this port. To flash the firmware, run
make flash. If you want to see the cameras' debug output, use
make debug right after
If you want the cameras to connect to a different SSID, modify the file
const char* WIFI_SSID = "camwifi"; const char* WIFI_PASS = "thisisacamwifi";
WiFi access point
You don't want to depend on any available wireless network not under your control because it could be unsecured. Furthermore, it has to be self-contained to allow users to test this demo. This is why there's a pod that creates a WiFi access point with the credentials mentioned in the above section:
SSID: camwifi Password: thisisacamwifi
If you want to expose a different SSID and use a different password, change the
Now, deploy this pod on MicroShift:
oc apply -f wifi-ap/cam-ap.yaml
After a few seconds, you should see it running:
oc get pods NAME READY STATUS RESTARTS AGE cameras-ap-b6b6c9c96-krm45 1/1 Running 0 3s
Looking at the pod logs, you can see how the process inside provides IP addresses to the cameras or any device connected to the wireless network.
[ Learn more about Data-intensive intelligent applications in a hybrid cloud blueprint. ]
The final step is deploying the AI models that perform face detection and face recognition. This pod is basically a Flask server that gets the streams of the cameras once they are connected and starts working on a discrete number of frames.
Deploy the AI models on MicroShift:
oc apply -f server/cam-server.yaml
After a few seconds:
oc get pods NAME READY STATUS RESTARTS AGE cameras-ap-b6b6c9c96-krm45 1/1 Running 0 4m37s camserver-cc996fd86-pkm45 1/1 Running 0 42s
You also need to create a service and expose a route for this pod:
oc expose deployment camserver oc expose service camserver --hostname microshift-cam-reg.local
MicroShift has mDNS built-in capabilities, and this route gets automatically announced, so the cameras can register to this service and start streaming video.
Looking at the
camserver logs, you can see this registration process:
oc logs camserver-cc996fd86-pkm45 * Serving Flask app 'server' (lazy loading) * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: off * Running on all addresses. WARNING: This is a development server. Do not use it in a production deployment. * Running on http://10.85.0.36:5000/ (Press CTRL+C to quit) [2022-01-21 11:18:46,203] INFO in server: camera @192.168.66.89 registered with token a53ca190 [2022-01-21 11:18:46,208] INFO in server: starting streamer thread [2022-01-21 11:19:34,674] INFO in server: starting streamer thread 10.85.0.1 - - [21/Jan/2022 11:19:34] "GET /register?ip=192.168.66.89&token=a53ca190 HTTP/1.1" 200 -
Finally, open a browser with the following URL:
This URL shows the feeds of all the cameras you've registered, and you can see how faces get detected.
[ Check out Red Hat's Portfolio Architecture Center for a wide variety of reference architectures you can use. ]
Running AI workloads at the edge is already happening. The challenge is managing thousands of these devices at scale and using the same tools you use in the cloud. MicroShift enables these cloud-native capabilities in small, field-deployed devices and brings an OpenShift-like experience to developers and administrators.
For more information, watch our presentation AI on the edge with MicroShift at DevConf.CZ 2022.