Difference between revisions of "Caspa camera boards"

From Gumstix User Wiki
Jump to: navigation, search
(add lens links)
 
(7 intermediate revisions by 3 users not shown)
Line 3: Line 3:
 
The MT9V032 sensor, at the heart of the Caspa FS camera board and the Caspa VL camera board, is not yet supported in the kernel, so a little work is necessary to get the camera up and running.
 
The MT9V032 sensor, at the heart of the Caspa FS camera board and the Caspa VL camera board, is not yet supported in the kernel, so a little work is necessary to get the camera up and running.
  
OMAP35x processors have dedicated hardware for capturing and processing data from image sensors. The Caspa camera sensor outputs raw 10-bit Bayer images which is transferred to the Image Signal Processor (ISP) via a parallel interface. The ISP contains various sub-modules that can be exported as Video for Linux (V4L2) devices in /dev. For additional hardware information see the external links section.
+
The OMAP3 processors in Gumstix' Overo COMs have dedicated hardware for capturing and processing data from image sensors. The Caspa camera sensor outputs raw 10-bit Bayer images which is transferred to the Image Signal Processor (ISP) via a parallel interface. The ISP contains various sub-modules that can be exported as Video for Linux (V4L2) devices in /dev. For additional hardware information see the external links section.
  
[[Image:Altoids.jpg|thumb|300px]]
+
Updated information about using the Caspa with the Overo is available [https://github.com/gumstix/yocto-manifest/wiki/Gstreamer-and-Caspa here]
  
 
== Quickstart ==
 
== Quickstart ==
  
 
=== Hardware ===
 
=== Hardware ===
 
[[Image:CaspaHardware.jpg|thumb|300px]]
 
 
 
Insert the white ribbon cable contact-side-up into the Caspa connector and contact side down into the Overo connector.
 
Insert the white ribbon cable contact-side-up into the Caspa connector and contact side down into the Overo connector.
  
 
Using a small flat-head screwdriver, loosen the lensholder set screw and rotate the lens to adjust the focus.
 
Using a small flat-head screwdriver, loosen the lensholder set screw and rotate the lens to adjust the focus.
  
=== Install Pre-built Kernel ===
+
=== Get Up and Running with an Image ===
  
The easiest way to get started is to replace the kernel and modules of an existing image. This has been tested on an Overo Fire with a Tobi expansion board using the desktop image available [http://www.sakoman.com/feeds/omap3/glibc/images/overo/201011150741/ here].
+
At the time of writing, streaming video with a Caspa from an Overo COM has been tested and confirmed working with the latest Overo images from our development branch. The links for the working image are:
  
Get the kernel and modules.
+
* [https://s3-us-west-2.amazonaws.com/gumstix-yocto/2014-05-21/overo/dev/MLO MLO]
$ wget http://cumulus.gumstix.org/images/angstrom/misc/caspapx/uImage-2.6.34
+
* [https://s3-us-west-2.amazonaws.com/gumstix-yocto/2014-05-21/overo/dev/u-boot.img U-Boot]
$ wget http://cumulus.gumstix.org/images/angstrom/misc/caspapx/modules-2.6.34.tgz
+
* [https://s3-us-west-2.amazonaws.com/gumstix-yocto/2014-05-21/overo/dev/uImage uImage]
 +
* Root Filesystems:
 +
** [https://s3-us-west-2.amazonaws.com/gumstix-yocto/2014-05-21/overo/dev/gumstix-console-image-overo.tar.bz2 Console]
 +
** [https://s3-us-west-2.amazonaws.com/gumstix-yocto/2014-05-21/overo/dev/gumstix-xfce-image-overo.tar.bz2 Desktop]
  
Copy to a bootable microSD card.
+
Instructions for using these files to get up and running are available at [http://gumstix.org/getting-started-guide.html gumstix.org].
$ tar -xf modules-2.6.34.tgz -C /media/mmcblk0p2/
+
$ cp uImage-2.6.34 /media/mmcblk0p1/uImage
+
$ cp uImage-2.6.34 /media/mmcblk0p2/boot/uImage
+
  
Alternatively, if you have a network connection, you can copy the files without removing the microSD.
+
Once you are up and running, the Caspa will appear as <pre>/dev/video6</pre>. You will need to run the following command to install the correct packages:
$ tar -xf modules-2.6.34.tgz
+
<pre>$ sudo smart install gstreamer gst-plugins-good-video4linux2 gst-plugins-bad-autoconvert gst-plugins-base-theora gst-plugins-good-rtp gst-plugins-good-udp</pre>
$ scp -r lib root@overo:/
+
$ scp uImage-2.6.34 root@overo:/boot/uImage
+
$ scp uImage-2.6.34 root@overo:/media/mmcblk0p1/uImage
+
$ ssh root@overo 'depmod; shutdown -r now'
+
  
=== Test ===
+
=== Testing ===
 +
==== RAW ====
 +
Run the following commands for a RAW capture:
  
From the serial console:
+
<pre>
 +
#Set the pipes
 +
media-ctl -r -l '"mt9v032 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
  
# export DISPLAY=:0.0
+
#Set the formats
 +
media-ctl -V '"mt9v032 3-005c":0[SGRBG10 752x480], "OMAP3 ISP CCDC":2[SGRBG10 752x480], "OMAP3 ISP preview":1[UYVY 752x480], "OMAP3 ISP resizer":1[UYVY 752x480]'
  
# gst-launch v4l2src ! xvimagesink
+
gst-launch -v v4l2src device=/dev/video6 ! rtpvrawpay ! udpsink host=[DESTINATION ADDRESS] port=5000
''or''
+
</pre>
# mplayer tv:// -tv driver=v4l2:device=/dev/video0
+
  
From the desktop:
+
You can then watch the video with the following command:
 +
<pre>
 +
gst-launch -v udpsrc port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)752, height=(string)480, colorimetry=(string)SMPTE240M, payload=(int)96" ! rtpvrawdepay ! xvimagesink
 +
</pre>
  
<left-click> Applications > Multimedia > Cheese
+
==== THEORA ====
 +
Set the pipes:
 +
<pre>
 +
#Set the pipes
 +
media-ctl -r -l '"mt9v032 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'
  
If video0 doesn't appear in /dev you might have to load the driver manually.
+
#Set the formats
 +
media-ctl -V '"mt9v032 3-005c":0[SGRBG10 752x480], "OMAP3 ISP CCDC":2[SGRBG10 752x480], "OMAP3 ISP preview":1[UYVY 752x480], "OMAP3 ISP resizer":1[UYVY 752x480]'
  
# depmod
+
LD_PRELOAD=/usr/lib/libv4l/v4l2convert.so gst-launch -v v4l2src device=/dev/video6 ! autoconvert ! theoraenc ! rtptheorapay ! udpsink host=[DESTINATION ADDRESS]
# modprobe mt9v032
+
</pre>
  
== Bitbake ==
+
Then you can watch the video:
  
You can build the kernel and modules using bit-bake. This allows you to incorporate the kernel with camera driver into an image of your design.
+
<pre>gst-launch -v udpsrc caps="[REPLACE WITH THE CAPS OUTPUT FROM ABOVE]" ! rtptheoradepay ! theoradec ! xvimagesink</pre>
  
$ cd ~/overo-oe/org.openembedded.dev
+
==== JPEG ====
$ git pull origin overo
+
A few extra packages are required for JPEG:
$ bitbake linux-omap3-caspapx
+
  
When the build completes, the kernel and modules will appear in ~/overo-oe/tmp/deploy/glibc/images/overo.
+
<pre>$ sudo smart install gst-plugins-base-videorate gst-plugins-good-jpeg gst-plugins-good-multipart gst-plugins-base-tcp</pre>
  
== Customization ==
+
Next you can begin capture:
 +
<pre>gst-launch v4l2src device=/dev/video6 ! autoconvert ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000 </pre>
  
This section describes how to customize the camera driver and kernel for your needs.
+
And finally view the result:
 +
<pre>gst-launch tcpclientsrc host=[IP ADDRESS OF YOUR GUMSTIX COM] port=5000 ! multipartdemux ! jpegdec ! xvimagesink</pre>
  
=== 2.6.34 ===
+
=== Display Frame Rate ===
 +
Replacing the last element <pre>xvimagesink</pre> with <pre>fpsdisplaysink</pre> shows load status overlayed on the screen. However the FPS remains at 0 and the number of dropped frame increases. Probably configuration issue.
 +
 
 +
 
 +
== Statistics ==
 +
These statistics were verified on an Overo AirSTORM COM:
 +
 
 +
{| class="wikitable"
 +
|
 +
! style="text-align:left;" | RAW
 +
! THEORA
 +
! JPEG
 +
|-
 +
! style="text-align:left;" | Bandwidth
 +
| ~10 MB/s
 +
| 30 KB/s
 +
| 130KB/s
 +
|-
 +
! style="text-align:left;" | Image Quality
 +
| High
 +
| Low
 +
| Medium
 +
|-
 +
! style="text-align:left;" | FPS
 +
|
 +
|
 +
|
 +
|-
 +
! style="text-align:left;" | CPU Load
 +
| ~94%
 +
| ~94%
 +
| ~94%
 +
|}
 +
 
 +
== Older Information ==
 +
 
 +
This section contains deprecated, but potentially useful information about using Caspa cameras.
 +
 
 +
 
 +
=== Kernel Version 2.6.34 Customizations ===
  
 
Get the kernel source that is used in the Gumstix kernel recipe (~/overo-oe/org.openmebedded.dev/recipes/linux/linux-omap3_2.6.34.bb) and create a new branch from the recipe SRCREV.
 
Get the kernel source that is used in the Gumstix kernel recipe (~/overo-oe/org.openmebedded.dev/recipes/linux/linux-omap3_2.6.34.bb) and create a new branch from the recipe SRCREV.
 +
$ git clone -b caspa https://github.com/gumstix/linux.git
 +
$ cd linux
 +
 +
Or, fetch and patch:
 
  $ cd ~
 
  $ cd ~
  $ git clone git://www.sakoman.com/git/linux-omap-2.6.git
+
  $ git clone git://www.sakoman.com/git/linux.git
  $ cd linux-omap-2.6
+
  $ cd linux
 
  $ git checkout -b 2.6.34 cb89736af28f583598e49a05249334a194d00f1d
 
  $ git checkout -b 2.6.34 cb89736af28f583598e49a05249334a194d00f1d
 
Get the patch.
 
 
  $ wget http://cumulus.gumstix.org/sources/mt9v032-2.6.34.patch
 
  $ wget http://cumulus.gumstix.org/sources/mt9v032-2.6.34.patch
 
  $ patch -p1 < mt9v032-2.6.34.patch
 
  $ patch -p1 < mt9v032-2.6.34.patch
Line 86: Line 133:
 
  $ cp mt9v032-2.6.34.defconfig .config
 
  $ cp mt9v032-2.6.34.defconfig .config
  
If you haven't done so yet, you'll need to [http://www.gumstix.net/Setup-and-Programming/view/Overo-Setup-and-Programming/Setting-up-a-build-environment/111.html set up your build environment]. Build the console image. This will ensure that the cross compiler, binutils, and libraries need to compile the kernel are installed.
+
If you haven't done so yet, you'll need to [http://www.jumpnowtek.com/gumstix/overo/Overo-Systems-with-Yocto.html set up your build environment]. Build the console image. This will ensure that the cross compiler, binutils, and libraries need to compile the kernel are installed.
 
  $ bitbake omap3-console-image
 
  $ bitbake omap3-console-image
  
Line 139: Line 186:
 
This is useful for adjusting the filters, white balance, and color blending coefficients used by the ISP preview module. The preview module is used for doing hardware conversion from 10-bit Bayer to YUV. If you are having color problems - especially under certain lighting conditions but not others - look at '''isppreview.c''' and the gamma correction tables in '''drivers/media/video/isp/'''
 
This is useful for adjusting the filters, white balance, and color blending coefficients used by the ISP preview module. The preview module is used for doing hardware conversion from 10-bit Bayer to YUV. If you are having color problems - especially under certain lighting conditions but not others - look at '''isppreview.c''' and the gamma correction tables in '''drivers/media/video/isp/'''
  
== Tuning ==
+
=== Tuning ===
  
=== Module Parameters ===
+
==== Module Parameters ====
 
By default, auto exposure, auto gain, and high dynamic range are all enabled. You can disable any or all of these to suit your needs by reloading the driver.
 
By default, auto exposure, auto gain, and high dynamic range are all enabled. You can disable any or all of these to suit your needs by reloading the driver.
  

Latest revision as of 16:26, 1 April 2016

The Caspa series of Camera Boards from Gumstix

The MT9V032 sensor, at the heart of the Caspa FS camera board and the Caspa VL camera board, is not yet supported in the kernel, so a little work is necessary to get the camera up and running.

The OMAP3 processors in Gumstix' Overo COMs have dedicated hardware for capturing and processing data from image sensors. The Caspa camera sensor outputs raw 10-bit Bayer images which is transferred to the Image Signal Processor (ISP) via a parallel interface. The ISP contains various sub-modules that can be exported as Video for Linux (V4L2) devices in /dev. For additional hardware information see the external links section.

Updated information about using the Caspa with the Overo is available here

Quickstart

Hardware

Insert the white ribbon cable contact-side-up into the Caspa connector and contact side down into the Overo connector.

Using a small flat-head screwdriver, loosen the lensholder set screw and rotate the lens to adjust the focus.

Get Up and Running with an Image

At the time of writing, streaming video with a Caspa from an Overo COM has been tested and confirmed working with the latest Overo images from our development branch. The links for the working image are:

Instructions for using these files to get up and running are available at gumstix.org.

Once you are up and running, the Caspa will appear as
/dev/video6
. You will need to run the following command to install the correct packages:
$ sudo smart install gstreamer gst-plugins-good-video4linux2 gst-plugins-bad-autoconvert gst-plugins-base-theora gst-plugins-good-rtp gst-plugins-good-udp

Testing

RAW

Run the following commands for a RAW capture:

#Set the pipes
media-ctl -r -l '"mt9v032 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'

#Set the formats
media-ctl -V '"mt9v032 3-005c":0[SGRBG10 752x480], "OMAP3 ISP CCDC":2[SGRBG10 752x480], "OMAP3 ISP preview":1[UYVY 752x480], "OMAP3 ISP resizer":1[UYVY 752x480]'

gst-launch -v v4l2src device=/dev/video6 ! rtpvrawpay ! udpsink host=[DESTINATION ADDRESS] port=5000

You can then watch the video with the following command:

gst-launch -v udpsrc port=5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)752, height=(string)480, colorimetry=(string)SMPTE240M, payload=(int)96" ! rtpvrawdepay ! xvimagesink

THEORA

Set the pipes:

#Set the pipes
media-ctl -r -l '"mt9v032 3-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3 ISP CCDC":2->"OMAP3 ISP preview":0[1], "OMAP3 ISP preview":1->"OMAP3 ISP resizer":0[1], "OMAP3 ISP resizer":1->"OMAP3 ISP resizer output":0[1]'

#Set the formats
media-ctl -V '"mt9v032 3-005c":0[SGRBG10 752x480], "OMAP3 ISP CCDC":2[SGRBG10 752x480], "OMAP3 ISP preview":1[UYVY 752x480], "OMAP3 ISP resizer":1[UYVY 752x480]'

LD_PRELOAD=/usr/lib/libv4l/v4l2convert.so gst-launch -v v4l2src device=/dev/video6 ! autoconvert ! theoraenc ! rtptheorapay ! udpsink host=[DESTINATION ADDRESS]

Then you can watch the video:

gst-launch -v udpsrc caps="[REPLACE WITH THE CAPS OUTPUT FROM ABOVE]" ! rtptheoradepay ! theoradec ! xvimagesink

JPEG

A few extra packages are required for JPEG:

$ sudo smart install gst-plugins-base-videorate gst-plugins-good-jpeg gst-plugins-good-multipart gst-plugins-base-tcp

Next you can begin capture:

gst-launch v4l2src device=/dev/video6 ! autoconvert ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000 

And finally view the result:

gst-launch tcpclientsrc host=[IP ADDRESS OF YOUR GUMSTIX COM] port=5000 ! multipartdemux ! jpegdec ! xvimagesink

Display Frame Rate

Replacing the last element
xvimagesink
with
fpsdisplaysink
shows load status overlayed on the screen. However the FPS remains at 0 and the number of dropped frame increases. Probably configuration issue.


Statistics

These statistics were verified on an Overo AirSTORM COM:

RAW THEORA JPEG
Bandwidth ~10 MB/s 30 KB/s 130KB/s
Image Quality High Low Medium
FPS
CPU Load ~94% ~94% ~94%

Older Information

This section contains deprecated, but potentially useful information about using Caspa cameras.


Kernel Version 2.6.34 Customizations

Get the kernel source that is used in the Gumstix kernel recipe (~/overo-oe/org.openmebedded.dev/recipes/linux/linux-omap3_2.6.34.bb) and create a new branch from the recipe SRCREV.

$ git clone -b caspa https://github.com/gumstix/linux.git
$ cd linux

Or, fetch and patch:

$ cd ~
$ git clone git://www.sakoman.com/git/linux.git
$ cd linux
$ git checkout -b 2.6.34 cb89736af28f583598e49a05249334a194d00f1d
$ wget http://cumulus.gumstix.org/sources/mt9v032-2.6.34.patch
$ patch -p1 < mt9v032-2.6.34.patch

Get the kernel configuration. This is the same one used for the linux-omap3-caspapx image.

$ wget http://cumulus.gumstix.org/sources/mt9v032-2.6.34.defconfig
$ cp mt9v032-2.6.34.defconfig .config

If you haven't done so yet, you'll need to set up your build environment. Build the console image. This will ensure that the cross compiler, binutils, and libraries need to compile the kernel are installed.

$ bitbake omap3-console-image

Add the cross compiler to your path and configure the kernel. The cross compiler location depends on the architecture of your build machine.

32-bit processors
$ export PATH=/home/<username>/overo-oe/tmp/sysroots/i686-linux/usr/armv7a/bin:${PATH} 
64-bit processors $ export PATH=/home/<username>/overo-oe/tmp/sysroots/x86_64-linux/usr/armv7a/bin:${PATH}

Build kernel and modules

$ make ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- uImage
$ make ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- modules

Install modules

$ mkdir ~/linux-omap-2.6/modules
$ make ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- INSTALL_MOD_PATH=./modules modules_install

Other Kernel Versions

The ISP code used for the 2.6.34 patch came from the arago-project. This is the OMAP3 PSP kernel integration/staging tree

$ git remote add git://arago-project.org/git/projects/linux-omap3.git aragoOMAP3 PSP kernel integration/staging tree
$ git fetch arago
$ git checkout -b arago

The V4L2 subdevice framework offers a way to configure the Image Signal Processor pipeline from user-space. This is currently under active development and users wishing to make use of this should visit the Media controller development repository here. This contains the bleeding edge Video for Linux media framework, ISP, and MT9V032 drivers.

MT9V032 Driver

If you want to take advantage of an unimplemented sensor feature, change defaults, etc., you need to modify the driver source. After you've made a change you can use a script like the following to quickly test your new driver.

#! /bin/bash 
TARGET_OVERO="root@10.0.1.15" make -j8 ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- modules ssh $TARGET_OVERO 'rmmod mt9v032' scp drivers/media/video/mt9v032.ko $TARGET_OVERO:/lib/modules/2.6.34/kernel/drivers/media/video/ ssh $TARGET_OVERO 'insmod /lib/modules/2.6.34/kernel/drivers/media/video/mt9v032.ko' ssh $TARGET_OVERO 'mplayer -display :0.0 tv:// -tv driver=v4l2:device=/dev/video0'

Image Signal Processor

There is some code that you can't modularize and modifying means rebuilding the kernel. Such is the case for the ISP code which is used by board-overo.c for hardware configuration. Another script like that in the previous section can be very useful.

This script rebuilds the kernel, copies it to the COM, and resets:

#! /bin/bash 
TARGET_OVERO="root@10.0.1.15" make -j8 ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- uImage; scp arch/arm/boot/uImage $TARGET_OVERO:/boot/uImage; scp arch/arm/boot/uImage $TARGET_OVERO:/media/mmcblk0p1/uImage ssh $TARGET_OVERO 'shutdown -r now'

This is useful for adjusting the filters, white balance, and color blending coefficients used by the ISP preview module. The preview module is used for doing hardware conversion from 10-bit Bayer to YUV. If you are having color problems - especially under certain lighting conditions but not others - look at isppreview.c and the gamma correction tables in drivers/media/video/isp/

Tuning

Module Parameters

By default, auto exposure, auto gain, and high dynamic range are all enabled. You can disable any or all of these to suit your needs by reloading the driver.

Get a list of driver parameters.

# modinfo mt9v032
filename:       /lib/modules/2.6.34/kernel/drivers/media/video/mt9v032.ko
license:        GPL
author:         Ignacio Garcia Perez <iggarpe@gmail.com>
description:    mt9v032 camera sensor driver
srcversion:     A46920FD64C35A2645E0D54
alias:          i2c:mt9v032
depends:        
vermagic:       2.6.34 mod_unload modversions ARMv7 
parm:           sensor_type:Sensor type: "color" or "mono" (charp)
parm:           auto_exp:Initial state of automatic exposure (int)
parm:           auto_gain:Initial state of automatic gain (int)
parm:           hdr:High dynamic range (int)
parm:           low_light:Enable companding (int)
parm:           hflip:Horizontal flip (int)
parm:           vflip:Vertical flip (int)

Depending on the light level, auto exposure might reduce the framerate as low as 15 frames per second. You can disable auto exposure like so:

# rmmod mt9v032;
# insmod /lib/modules/2.6.34/kernel/drivers/media/video/mt9v032.ko auto_exp=0

If this makes your image too dark, you can disable auto gain and enable companding mode (more information here).

# rmmod mt9v032
# insmod /lib/modules/2.6.34/kernel/drivers/media/video/mt9v032.ko auto_exp=0 auto_gain=0 low_light=1


External Links

Caspa VL (filtered lens): V-4303.6-1


Caspa FS (unfiltered lens): KLB-0360


OMAP35x Technical Reference Manual

MT9V032 Datasheet

Caspa layout and schematic files