"... and no one shall work for money, and no one shall work for fame; But each for the joy of the working, and each, in his separate star, shall draw the thing as he sees it, for the god of things as they are"
-Kipling
-Kipling
FPath is a project to explore the possibilities of the Feynman Path to Nanotechnology. Essentially this means using tools to make small tools which then make smaller tools. See the main FPath Project page for more details.
This page documents FPath Experiment 002 (FPath_Ex002). The purpose of this experiment is to develop the ability to inject video images into the webcam stream processed by the Walnut software and to get that software to react to those images as if the virtual entity actually existed in the real world.
In order to properly demonstrate the image injection capabilities, the target goal of this experiment is to:
Move a colored square on a rotating platform as close as possible to a virtual static square of a different color.
This experiment builds on the results of FPath Experiment 001 which developed the hardware and software to implement the same goal but with an actual physical static target square.
The physical infrastructure used is the same as that in Experiment 001 and only modest changes were required to the Walnut software. The starting point for this experiment is Walnut Server/Client version 00.02.03 and the changes made have been released as version 00.02.04.
Since the hardware and most of the software are identical to Experiment 001 they will not be described in detail here. Have a look at the documentation of that experiment for more information.
Below is a specific list of the steps involved in reaching the goal.
The experiment was successful. The virtual green square was injected into the webcam stream so that it appeared at various points on the screen. The Walnut Software reacted to the appearance of the square (as in Experiment 001) as if it were a physical entity. The changes made to the software to place a bmp or transparent png on the screen also worked well.
Video of using a virtual entity as an image recognition target with PID Control
This experiment is now complete. The Walnut software associated with this experiment can be found under Commit ID: d9a1b38 on the GitHub repo.
The ability to inject virtual entities into a webcam stream and have the system react to it as if it were real should prove to be useful in the future. For example, in the image at left, a sample circuit has been overlayed on the display. Robot manipulators could use this network of colored paths as a goal or as real world position markers. Objects could be built by moving items onto or around the virtual traces or the traces could be used as paths for movement. Since the virtual entities do not actually exist in the real world they could be dynamically adjusted so as to change the system goals or paths. In addition, semi-transparent traces would change color as objects were moved onto or through it. Since the image recognition system detects color as well as shape, this effect could serve as an automatic real-world marker of the state of the target environment. In other words, a component off of the trace is one color and a component at the same location as the trace is another color as it is being viewed through a semi-transparent colored overlay. For the purposes of object recognition, the automatic color change is indicative of the location state. No record of an objects position would need to be kept - the real world state of the system is the record.
The intellectual property rights to all new and/or original ideas and technologies documented under the FPath project and sub-projects are claimed in full by the author and are immediately released into the public domain under the terms of the MIT License. Any ideas, techniques, processes or methods of work documented in the FPath project and sub-projects must be considered to be prior art and must be cited in any patent applications.
The contents of the FPath project and sub-projects are provided "as is" without any warranty of any kind and without any claim to accuracy. Please be aware that the information provided may be out-of-date, incomplete, erroneous or simply unsuitable for your purposes. Any use you make of the information is entirely at your discretion and any consequences of that use are entirely your responsibility.