- Vision Sensing.jpg (8.84 KiB) Viewed 20340 times
Previously, electro-optical sensory systems were quite expensive and were mostly found in military hardware such as the AGM-65 Maverick missile and similar systems deployed around the world. Now however, affordable and capable vision sensing devices are available to the rest of us. One such device is the Pixy2 camera which is capable of line following and object recognition. Available for around $60 US, Pixy2 is a great addition to small, capable platforms such as fitlet2.
Older Machine Vision Sensing Technology - Expensive and Limited to Military Applications
- Maverick.jpg (30.48 KiB) Viewed 20340 times
Pixy2 - Modern Machine Vision Sensing Technology - Affordable and Available to All
- Pixy2.jpg (46.29 KiB) Viewed 20340 times
About Pixy2
Pixy2 is the latest incarnation of Pixy which was born of a partnership between the Carnegie Mellon Robotics Institute and Charmed Labs. The hardware and software utilized in Pixy2 are open source, so it is an ideal device for use by the makers among us.
- Pixy2 Open Source.jpg (8.58 KiB) Viewed 20337 times
Pixy2 has a couple of sensing modes that are quite useful for the maker - especially for robotics applications. These modes are:
1. Object Detection and Recognition
Pixy2 learns and detects objects based on colors using an algorithm known as Color Connected Components. Color based object recognition was utilized because of its efficiency, speed, and relatively "bullet-proof" performance. Pixy2 has the ability to simultaneously detect and track a large number of objects simultaneously. This is a capability that was once reserved for only very complex and expensive military systems.
- Orange.png (126.91 KiB) Viewed 20340 times
Pixy2 has the ability to track and follow lines using its custom line tracking algorithm. Unlike photodetector based solutions, Pixy2's vision based line tracking solution has the ability to anticipate by looking ahead and the ability to easily identify and navigate through intersections.
- Line Tracking 1.png (42.06 KiB) Viewed 20340 times
Hardware Used
- Hardware.jpg (26.33 KiB) Viewed 20340 times
1. fitlet2
2. Pixy2 camera system
3. Pixy2 Pan-Tilt kit
Software
The basic software package for configuring and experimenting with Pixy2 is PixyMon. It is available for Linux, Mac, and Windows. This app note focuses on the Linux version. The preferred distro for the Linux version of PixyMon is Linux Mint which is what I use.
- LM Friendly.jpg (6.85 KiB) Viewed 20337 times
Detailed instructions for installing PixyMon in Linux are found here:
https://docs.pixycam.com/wiki/doku.php? ... n_on_linux
In my case I installed PixyMon in /usr/local/src/pixy2 in order to adhere to what I perceive to be standard Linux practice - although there are many different opinions concerning what is standard practice in Linux.
Also, I wrote a short script PixyMon.sh which I made executable and placed in /home/[username]/bin for universal access.
PixyMon.sh
#!/bin/bash
# Script to Run PixyMon
cd /
./usr/local/src/pixy2/build/pixymon/PixyMon
Test Results
Pixy2 in conjunction with fitlet2 proved to be a fun and interesting system to explore. I was impressed with Pixy2's ability to accurately and consistently detect, identify, and track objects. The following images and videos provide more detail.
1. Object Detection, Recognition, and Tracking
Basic Shape/Color Recognition - Primary Colors
- Primary.jpg (143.77 KiB) Viewed 20337 times
- Secondary.jpg (95.87 KiB) Viewed 20337 times
Basic Shape/Color Recognition - Multiple Objects
- Multi.jpeg (150.4 KiB) Viewed 20337 times
Basic Shape/Color Tracking - Pixy2 Pan/Tilt Kit - Ripe Tomato
2. Line Tracking
Since I am using Pixy2 in a more or less static configuration, I was not able to make a detailed exploration of its line tracking performance. However, the following picture shows that Pixy2 does its best to search its field of view for lines to follow.
- Line Tracking.png (65.52 KiB) Viewed 20337 times
Potential Projects
1. Lunar Tracker for aiming "Moon Bounce" laser and microwave transmissions.
2. Laser tracker for mobile platform.
3. Automated fruit/vegetable harvesting system using color/shape recognition.
4. Vehicle "road ahead" hazard detection, recognition, and warning system.
5. Etc.
Actual Line Following Project (By Others)
Personally, I think a fitlet2 would make a fine field-hardened replacement for the laptop used in the following video.
Resources for Developers, Experimentors, and Makers
APIs, Libraries, and Software
https://pixycam.com/downloads-pixy2/
Wiki
https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:start
Forums
https://forum.pixycam.com/
Where Do fitlet2 and Pixy2 Go From Here?
- Where To.jpeg (4.17 KiB) Viewed 20337 times