PLEASE READ: this tutorial assumes that the Dragonfly Java App has been already setup on the CLIENT machine following the instructions available starting from this page.

Any USB camera connected to a machine running macOS or Linux (from now on SOURCE machine) can easily be used as a remote camera for Dragonfly streaming a video over an Internet connection to another machine running Dragonfly (from now on CLIENT machine).

1- Calibration of the camera

  1. If not already done please setup the Dragonfly Java App on the CLIENT machine following carefully the instructions available starting from this page.
  2. Launch the Dragonfly Java App on the CLIENT machine.
  3. Connect the USB camera to the CLIENT machine (not to the SOURCE machine!).
  4. Click on the Configuration tab and set the CAM_SOURCE parameter to -1.
  5. Click on Save and Restart.
  6. Click on the Calibration tab of the Dragonfly Java App.
  7. Download the calibration pattern at this link.
  8. Open it on the screen of your PC or print it out on an A4 or Letter paper and place it on a plain table/desk or stick the calibration pattern on a cardboard cover. Do not perform the calibration with wobbly paper in your hand. Those results are unusable.
  9. Measure the distance in millimeters (NO inches and NO centimeters!) of the HDIST line visible inside the calibration pattern.
  10. Configure the distance just measured inside the field HDIST.
  11. Select 640 x 480 as Video Resolution of your camera from the drop down menu.
  12. ENABLE the Fisheye Lens if you are using a camera with a FOV greater than 120 degrees.
  13. ENABLE assisted mode.
  14. Start the calibration by clicking on START.
  15. The server presents you a  pulsating color pattern overlay, surrounded by a black frame with a positioning indicator in the top-left corner.
  16. If you hold your calibration pattern plate into the camera, you should see the black dots being connected by colored lines.
  17. Your challenge is now to match the pattern displayed on the screen with your calibration pattern plate. The snapshots are made automatically.
  18. It doesn’t matter whether you move the camera over the calibration pattern or the calibration pattern in front of the camera.
  19. The server will present at least 21 poses, which all need to be matched. TIP: if it helps you to coordinate your movements, you can have the image displayed mirrored by clicking on the button Flip View.
  20. After you have finally made it, the server automatically start the computation of the calibration parameters of your camera and will be notified once this process is finished.
  21. At the end of the calibration process:
    • click on the Download button and download the JSON calibration file generated by the calibration tool.
    • place the JSON calibration file inside the config folder of the Dragonfly Java App (cd dragonfly_linux/application/Dragonfly2Java_jar/config).
    • set the CAM_CALIBRATION_FILE parameter under the Configuration tab to the name of the JSON calibration file placed inside the config folder.
    • click on Save and Restart.

2 – Setup of the SOURCE machine

2.1 – Install Gstreamer

We use Gstreamer to stream the video from the SOURCE machine. To install it enter the following commands on your SOURCE machine:

sudo apt-get -y install libgstreamer1.0-dev
sudo apt-get -y install libgstreamer-plugins-base1.0-dev
sudo apt-get -y install gstreamer1.0-libav
sudo apt-get -y install gstreamer1.0-plugins-bad
sudo apt-get -y install gstreamer1.0-tools
sudo apt-get -y install gstreamer1.0-plugins-good
sudo apt-get -y install gstreamer1.0-plugins-ugly
sudo apt-get -y install gstreamer1.0-nice

2.2 – Activate the streaming

On the SOURCE machine:

  1. Connect the USB camera (the one that you have just calibrated using the CLIENT machine).
  2. Launch the following command to list all video devices picked up by Ubuntu in order to find the ID of the USB camera:
    sudo apt-get install v4l-utils
    v4l2-ctl --list-devices
  3. Launch one of the following commands to activate the video stream (ip_of_your_device is the local IP address assigned to the SOURCE machine):
    1. MJPEG streaming – use this solution if the hardware does not allow live H264 encoding, or if H264 is not satisfying:
      gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,width=640,height=480,framerate=30/1' ! decodebin name=dec ! queue ! videoconvert ! jpegenc ! queue ! multipartmux ! tcpserversink host=ip_of_your_device port=5000
    2. H264 streaming much lower bandwidth requirements than MJPEG, but requires live encoding and some CPUs might not be powerful enough.
      gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,width=640,height=480,framerate=30/1' ! videoconvert ! x264enc tune=zerolatency byte-stream=true ! rtph264pay ! gdppay ! tcpserversink host=ip_of_your_device port=5000 sync=false
    3. H264 (accelerated) streaming – same as H264 streaming, but does not require a powerful CPU encoding. However, all platforms are not compatible. Raspberry Pi and Jetson Nano should work fine.
      gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,width=640,height=480,framerate=30/1' ! videoconvert ! omxh264enc ! rtph264pay ! gdppay ! tcpserversink host=ip_of_your_device port=5000 sync=false

3 – Make the video stream available to the Dragonfly Java App

  1. If not already active, launch the Dragonfly Java App on the CLIENT machine.
  2. Click on the Configuration tab:
    1. set the CAM_SOURCE parameter to one of the following strings depending on the type of streaming activated on the SOURCE machine (ip_of_your_device is the local IP address assigned to the SOURCE machine):
      • For MJPEG streaming:
        gstreamer:tcpclientsrc host=ip_of_your_device port=5000 ! multipartdemux ! image/jpeg, framerate=30/1 ! jpegparse ! jpegdec ! videoconvert ! appsink sync=false
      • For H264 streaming:
        gstreamer:tcpclientsrc host=ip_of_your_device port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! appsink sync=false
      • For H264 (accelerated) streaming:
        gstreamer:tcpclientsrc host=ip_of_your_device port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! appsink sync=false
    2. set the CAM IMAGE WIDTH and HEIGHT to 640 x 480.
  3. Click on Save and Restart.

Your USB camera connected to the SOURCE machine can now be used as a remote streaming device for the Dragonfly instance running on the CLIENT machine. All the information about the Dragonfly Java App can be found inside this page.