"... and no one shall work for money, and no one shall work for fame; But each for the joy of the working, and each, in his separate star, shall draw the thing as he sees it, for the god of things as they are"
-Kipling
-Kipling
FPath is a project to explore the possibilities of the Feynman Path to Nanotechnology. Essentially this means using tools to make small tools which then make smaller tools. See the main FPath Project page for more details.
This page documents FPath Experiment 001 (FPath_Ex001). The fundamental purpose of this experiment was to build and test the various software and hardware components and get them all to work together. A secondary purpose was to get some experience coding a PID style motion controller in C#.
Of course, rather than vague handwaving aspriations, an experiment should have a specific goal - even if you never reach it. This provides a definitive stop point and the work undertaken to reach that goal should fulfil the fundamental purpose. In this experiment the target goal is to:
Move a colored square on a rotating platform as close as possible to a static square of a different color.
Doesn't seem like much does it? Especially since you, as a human, could easily perform such an operation. Well, actually, it's a bit more tricky than it seems to get a computer to do it. To automate this action you need a webcam to image the the two squares and software which can take that stream of images and recognize the positions of those squares. The orientation of the colored square on the rotating platform (dynamic square) changes as the platform rotates so the image recognition algorythm needs to be able to cope with that. Once the squares are identified, the software needs to somehow activate the motor of the rotating platform and then stop moving it once the squares are as close together as they can be. It should be noted that the center of the rotating platform is assumed not to be known. This means simple math cannot be used to drive the dynamic square as close as possible to the square that does not move (the static square). Instead a PID style algorythm must be used to minimize the distance.
The Walnut Software provides the server and client software for the system. The starting point for this experiment is Walnut Server/Client version 00.02.02 and the changes made have been released as version 00.02.03.
As of Walnut v00.02.02 the Walnut Server software can accept a stream of images from a webcam and identify squares and circles within the image stream. The Walnut server software can, optionally, record the image stream to disk as an MP4 file which is useful for documenting the results. The Walnut Client is fully capable of accepting information from the Walnut Server and also interacting with an assembly language program running in the Programmable Realtime Units (PRU's) of a Beaglebone Black (BBB) to output a stream of STEP and DIR pulses suitable for driving a stepper motor.
Below is a specific list of the steps involved in reaching the goal.
The experiment was successful. After some considerable adjustment, the hardware and software performed flawlessly. The video in the link below shows the platform rotating in order to move the red square as close to the green square as possible. If the static green square target is moved, the software will adjust and track the dymamic red square to the new position. Note that the speed of the movement is not constant and that there is not too much overshoot and backtracking as the dynamic square approaches the static square.
Video of moving a square close to another square via image recognition with PID Control
This experiment is now complete. Since the PID style algorythm developed is not specifically required for future projects it will not be developed or improved further until such time as it is needed. The Walnut software associated with this experiment can be found under Commit ID: 4902917 on the GitHub repo.
The platform itself is quite simple. An old RepRap Mendel 3D printer was pressed into service as the structure (click on the thumbnail at left for a bigger image). None of the existing stepper motors and carriage drives were used - although they were left in place for possible future use. A hole was drilled in the center of the 6mm MDF bed and a generic geared bipolar stepper motor was attached to the underside. A geared stepper motor was chosen since it gives far more "steps per revolution" than the 200 or so steps one might obtain from an ungeared version and this extra resolution enabled the PID style algorythm to work much better. The rotating disk was cut out of an old cereal box and both the bed and disk were covered with a sheet of white printer paper. The disk was attached to the shaft of the stepper motor with cyanoacrylate glue.
The squares measured roughly 10mm square and were cut out of old paint swatch samples. Paint swatches are a very good way to get numerous examples of pretty much any desired color for free.
The web camera is a Logitech C920 USB web cam plugged into a Windows 10 PC. The autofocus did not work especially well and the Logitech supplied utility software was used to manually focus the webcam on the bed. The resolution and format of the webstream used was 640x480 YUY2. The C920 is, of course, capable of much finer resolutions than that but 640x480 was sufficient.
The C920 camera viewed the bed through a commonly available and inexpensive LED ring light typically sold so Internet influencers can better light their cell phone videos. This LED ring light had adjustable intensity and somewhat adjustable color values (harsh white to warm white).
Ultimately the Beaglebone Black can only output STEP and DIR signals and the bipolar stepper motor needs appropriate power levels on it's two coils. Accordingly, a DRV8834 Low-Voltage Stepper Motor Driver Carrier from Pololu was used to provide this conversion. The DRV8834 driver was mounted on an isolation routed single sided PCB board which contained a few other passive components such as quick release terminal block connectors, a LED and a barrel jack for the low voltage, low current DC supply for the stepper motor. The board was designed in KiCad version 8 and the LineGrinder (version 03.07) open source software was used to convert the KiCad Gerber files into GCode in order to isolation route the board.
The STEP and DIR signals are produced by the assembly code running in the PRU's on the Beaglebone Black. They appear on the pins 45 and 46 (respectively) of the P8 Header. In the BBB the Device Tree Overlays and assembly language code are configured to support this. The 3.3v signal power (as opposed to stepper motor power) of the driver board are derived from pins 1 and 3 of the P8 header on the Beaglebone Black.
The goal of the project was to move a dynamic square of one color as close as possible to a static square of another color. The coordinates of the two squares are discovered via image recognition (see the Prism Project) out of the webcam stream (see the Tanta Project) by the Walnut Server software running on Windows 10 and transmitted via Ethernet to the Walnut Client software running on a Beaglebone Black microcontroller. The server information arrives imbedded in a fully typed class (see the RemCon project) which appears fully instantiated in the Walnut Client. The arrival of the incoming data packet triggers a routine in the Walnut client (in a separate thread) that calls a function named SetWaldosFromServerClientData(). For this experiment, the SetWaldosFromServerClientData() function assumes the location information of the colored squares is the only data available and calls a subroutine named Behaviour_MoveClose(). The Behaviour_MoveClose() takes as inputs two coordinates (the locations of the colored squares) and outputs a value indicating the speed and direction. The Behaviour_MoveClose() code is intended to be one of a suite of standardised behaviours. However, this mechanism is not fully formed in the current Walnut Client version and the call is essentially just hard coded.
The output of the Behaviour_MoveClose() code is directly used to set values which are picked up by assembly language software running in the Programable Realtime Unit (PRU) of the BeagleboneBlack. The PRU is essentially a CPU within a CPU and its behaviour is entirely deterministic since it runs only one process and has no capability to swap processes in and out. This is extremely useful if you need absolutely realtime outputs or inputs. The PRU1_StepperIO.p assembly language program is autocompiled when the Walnut client is compiled and is designed to produce consistently timed waveforms for up to six stepper motors (see the Tilo Project). It does not matter for the timings how many steppers are active or if they are dynamically added or removed. In effect, this means all the Walnut client has to do is set a value for the speed and direction and the assembly language code running in the PRU will pick it up (they share an address space) and generate the appropriate signals.
The signals output by the PRU are transmitted to the Stepper Motor Driver board (see above) and that hardware drives the stepper at the specified speed and direction. The moving stepper motor turns the turntable and the dynamic square attached to it changes position. Of course, the changing position is seen by the webcam, processed by the Walnut Server software and the new coordinates transmitted to the Walunt Client software. Thus the loop is closed and, as the new coordinates are fed into the Behaviour_MoveClose() function, new values of speed and direction are generated.
The algorithm in the Behaviour_MoveClose() is not stateless. Past coordinate values, positions and speeds are known to it and are considered when deciding on the new values. Thus the code functions like a PID controller in this respect although this particular implementation is much simpler.
The intellectual property rights to all new and/or original ideas and technologies documented under the FPath project and sub-projects are claimed in full by the author and are immediately released into the public domain under the terms of the MIT License. Any ideas, techniques, processes or methods of work documented in the FPath project and sub-projects must be considered to be prior art and must be cited in any patent applications.
The contents of the FPath project and sub-projects are provided "as is" without any warranty of any kind and without any claim to accuracy. Please be aware that the information provided may be out-of-date, incomplete, erroneous or simply unsuitable for your purposes. Any use you make of the information is entirely at your discretion and any consequences of that use are entirely your responsibility.