Since one of my goals for my Measy U2C stick is to use it as an in-home security camera, thanks to its integrated microphone and webcam (model GC2035 from GalaxyCore), I had to find a way to support them in Picuntu Linux (the RK3066 based Xubuntu 12.10 from developer Alok Sinha).
Wouldn't it serve its purpose running headless (no monitor) and attached to this battery or to a phone charger, ready to send an email to your phone upon sensing too much noise or "seeing" somebody? and we are talking of a CPU with Video encoding capabilities!
So, there is one of the motivations, now onto the quest. The first problem was that Picuntu does not include a driver for the GC2035 CMOS sensor (and the GC2015 it supports is different enough not to be fine), so I set out to github.com and its awesome code search just to find the latest RK3066 kernel (for Android Jelly Bean support), some months more modern than the one Picuntu uses.
Since porting Picuntu Linux to the JB (Jelly Bean) RK3066 kernel is taking me some time, at least I have made enough changes to successfully port the GC2035 driver.
 |
| Note the "(GC2035)" below the CMOS sensor at the right |
I believe the Kimdecent B12 and, probably, the NX003II sticks are clones of the Measy U2C, in which case this solution would also work for them. If you are the owner of an RK3066 stick with integrated camera and are willing to take high resolution pictures of both sides of your PCB, we could all confirm that.
To add the driver to your Picuntu Linux stick, you have to recompile the kernel (quite easy actually) following the steps in this post but in the step where you download the latest Picuntu source code:
git clone https://github.com/aloksinha2001/picuntu-3.0.8-alok.gityou have to instead download the updated source code*:
git clone git://github.com/Galland/picuntu-3.0.8-alok.gitThe rest of the instructions are exactly the same and you can still use the same SD card installation of Picuntu that you already have, since it doesn't change (only the kernel that goes inside the stick's flash recovery partition).
You'll just find that, upon bootup, the gc2035 driver is loaded and attached as a front camera (front_2) and the new device appears as:
/dev/video0
[*] You can review the changes by parsing through my commit history or just know that I've mostly added the GC2035 driver, updated the machine config to "talk" to this CMOS sensor and added a few notable bugfixes from Rockchip's JB kernel.
The catch
While the system can indeed communicate with the sensor, I've found that the driver is able to output NV12, NV21, NV16, or NV61 Video4Linux2 pixel formats (the way in which the luminance and chrominance samples of pixels are set up in the buffer where a captured image is sent to apps).
This would be fine unless for the fact that I haven't found an application able to get that V4L2 format and save an image to disk from it, though I haven't searched that much, I must say (see below).
While I temporarily move on to other tasks in this Linux support "challenge", this is a part where you, dear Measy U2C or clone owner, could be of great help, by googling and testing to get this very last step through! :-)
Even if you only find an app that dumps the raw NV* capture to a file, I can very quickly write a tool to transform it to a viewable RGB format.
My brief findings
Using the app "fswebcam" the output is the following:
user@picuntu:~$ fswebcam -v output.jpg
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
src_v4l2_get_capability,87: /dev/video0 information:
src_v4l2_get_capability,88: cap.driver: "rk-camera-rk30"
src_v4l2_get_capability,89: cap.card: "gc2035_front_2-270"
src_v4l2_get_capability,90: cap.bus_info: ""
src_v4l2_get_capability,91: cap.capabilities=0x04000001
src_v4l2_get_capability,92: - VIDEO_CAPTURE
src_v4l2_get_capability,103: - STREAMING
No input was specified, using the first.
src_v4l2_set_input,181: /dev/video0: Input 0 information:
src_v4l2_set_input,182: name = "Camera"
src_v4l2_set_input,183: type = 00000002
src_v4l2_set_input,185: - CAMERA
src_v4l2_set_input,186: audioset = 00000000
src_v4l2_set_input,187: tuner = 00000000
src_v4l2_set_input,188: status = 00000000
src_v4l2_set_pix_format,541: Device offers the following V4L2 pixel formats:
src_v4l2_set_pix_format,554: 0: [0x3231564E] 'NV12' (YUV420 NV12)
src_v4l2_set_pix_format,554: 1: [0x3631564E] 'NV16' (YUV422 NV16)
src_v4l2_set_pix_format,554: 2: [0x3132564E] 'NV21' (YUV420 NV21)
src_v4l2_set_pix_format,554: 3: [0x3136564E] 'NV61' (YUV422 NV61)
src_v4l2_set_pix_format,554: 4: [0x3231564E] 'NV12' (YUV420 NV12)
src_v4l2_set_pix_format,554: 5: [0x3631564E] 'NV16' (YUV422 NV16)
src_v4l2_set_pix_format,554: 6: [0x3132564E] 'NV21' (YUV420 NV21)
src_v4l2_set_pix_format,554: 7: [0x3136564E] 'NV61' (YUV422 NV61)
Unable to find a compatible palette format.
Whereas VLC tries to access it in four different ways, unsuccessfully (notice the warnings):
UPDATE (2013/04/05): It seems the previous problems related to I/O mmap were due to the kernel having a too small value for CONSISTENT_DMA_SIZE, which I have already changed from 2 to 8 MB in the GitHub repository above. Still no video but the warnings are:
v4l2 warning: Could not select any of the default chromas; attempting to open as MPEG encoder card (access)
v4l2 warning: Could not select any of the default chromas; attempting to open as MPEG encoder card (access)
ps warning: this does not look like an MPEG PS stream, continuing anyway
ps warning: garbage at input, trying to resync...
Ideas welcome :)
Message log output before update:
main debug: adding item `v4l2:///dev/video0' ( v4l2:///dev/video0 )
qt4 debug: Adding a new MRL to recent ones: v4l2:///dev/video0
main debug: no fetch required for (null) (art currently (null))
main debug: rebuilding array of current - root Lista de reproducción
main debug: rebuild done - 1 items, index -1
main debug: processing request item: v4l2:///dev/video0, node: null, skip: 0
main debug: resyncing on v4l2:///dev/video0
main debug: v4l2:///dev/video0 is at 0
main debug: starting playback of the new playlist item
main debug: resyncing on v4l2:///dev/video0
main debug: v4l2:///dev/video0 is at 0
main debug: creating new input thread
main debug: Creating an input for 'v4l2:///dev/video0'
main debug: using timeshift granularity of 50 MiB, in path '/tmp'
main debug: `v4l2:///dev/video0' gives access `v4l2' demux `' path `/dev/video0'
main debug: creating demux: access='v4l2' demux='' location='/dev/video0' file='/dev/video0'
main debug: looking for access_demux module: 1 candidate
v4l2 debug: opening device '/dev/video0'
qt4 debug: IM: Setting an input
v4l2 debug: trying kernel V4L2
v4l2 debug: device gc2035_front_2-270 using driver rk-camera-rk30 (version 0.2.14) on
v4l2 debug: the device has the capabilities: 0x04000001
v4l2 debug: (X) Video Capture, ( ) Audio, ( ) Tuner, ( ) Radio
v4l2 debug: ( ) Read/Write, (X) Streaming, ( ) Asynchronous
v4l2 debug: video input 0 (Camera) has type: External analog input *
v4l2 debug: input set to 0
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: found default width and height of 640x480
v4l2 debug: will try to find optimal width and height
v4l2 warning: Could not select any of the default chromas; attempting to open as MPEG encoder card (access)
v4l2 debug: trying library V4L2
v4l2 debug: device gc2035_front_2-270 using driver rk-camera-rk30 (version 0.2.14) on
v4l2 debug: the device has the capabilities: 0x05000001
v4l2 debug: (X) Video Capture, ( ) Audio, ( ) Tuner, ( ) Radio
v4l2 debug: (X) Read/Write, (X) Streaming, ( ) Asynchronous
v4l2 debug: video input 0 (Camera) has type: External analog input *
v4l2 debug: input set to 0
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: device supports chroma RV24 [RGB3, RGB3]
v4l2 debug: device supports chroma RV24 [BGR3, BGR3]
v4l2 debug: device supports chroma I420 [YU12, YU12]
v4l2 debug: device supports chroma YV12 [YV12, YV12]
v4l2 debug: found default width and height of 640x480
v4l2 debug: will try to find optimal width and height
v4l2 warning: Could not select any of the default chromas; attempting to open as MPEG encoder card (access)
main debug: no access_demux module matching "v4l2" could be loaded
main debug: TIMER module_need() : 885.952 ms - Total 885.952 ms / 1 intvls (Avg 885.952 ms)
main debug: creating access 'v4l2' location='/dev/video0', path='/dev/video0'
main debug: looking for access module: 1 candidate
v4l2 debug: opening device '/dev/video0'
v4l2 debug: trying kernel V4L2
v4l2 debug: device gc2035_front_2-270 using driver rk-camera-rk30 (version 0.2.14) on
v4l2 debug: the device has the capabilities: 0x05000001
v4l2 debug: (X) Video Capture, ( ) Audio, ( ) Tuner, ( ) Radio
v4l2 debug: (X) Read/Write, (X) Streaming, ( ) Asynchronous
v4l2 debug: video input 0 (Camera) has type: External analog input *
v4l2 debug: input set to 0
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: device supports chroma RV24 [RGB3, RGB3]
v4l2 debug: device supports chroma RV24 [BGR3, BGR3]
v4l2 debug: device supports chroma I420 [YU12, YU12]
v4l2 debug: device supports chroma YV12 [YV12, YV12]
v4l2 debug: found default width and height of 640x480
v4l2 debug: will try to find optimal width and height
v4l2 debug: Driver requires at most 460800 bytes to store a complete image
v4l2 debug: Interlacing setting: progressive
v4l2 error: mmap failed: Cannot allocate memory
v4l2 debug: trying library V4L2
v4l2 debug: device gc2035_front_2-270 using driver rk-camera-rk30 (version 0.2.14) on
v4l2 debug: the device has the capabilities: 0x05000001
v4l2 debug: (X) Video Capture, ( ) Audio, ( ) Tuner, ( ) Radio
v4l2 debug: (X) Read/Write, (X) Streaming, ( ) Asynchronous
v4l2 debug: video input 0 (Camera) has type: External analog input *
v4l2 debug: input set to 0
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: device codec NV12 (YUV420 NV12) not supported
v4l2 debug: device codec NV16 (YUV422 NV16) not supported
v4l2 debug: device codec NV21 (YUV420 NV21) not supported
v4l2 debug: device codec NV61 (YUV422 NV61) not supported
v4l2 debug: device supports chroma RV24 [RGB3, RGB3]
v4l2 debug: device supports chroma RV24 [BGR3, BGR3]
v4l2 debug: device supports chroma I420 [YU12, YU12]
v4l2 debug: device supports chroma YV12 [YV12, YV12]
v4l2 debug: found default width and height of 640x480
v4l2 debug: will try to find optimal width and height
v4l2 debug: Driver requires at most 460800 bytes to store a complete image
v4l2 debug: Interlacing setting: progressive
v4l2 error: device does not support mmap I/O
main debug: no access module matching "v4l2" could be loaded
main debug: TIMER module_need() : 226.883 ms - Total 226.883 ms / 1 intvls (Avg 226.883 ms)
main error: open of `v4l2:///dev/video0' failed
main debug: finished input
main debug: dead input
main debug: changing item without a request (current 0/1)
main debug: nothing to play
qt4 debug: IM: Deleting the input
BTW, this article has been posted with Picuntu (Xubuntu) on the Measy U2C
I've managed to access a rockchip-based camera on the rk3026 SOC. I guess that some of my findings may be useful to you as well (better late than never :)
ReplyDeleteThe main problem is that the rockchip driver seems support only V4L2_MEMORY_OVERLAY. At least that's how android uses it. I haven't found any real documentation about this mode. Most other V4L apps use V4L2_MEMORY_MMAP.
I've hacked some of the camera driver code to make it work, you can see my patch here: pastie.org/9452632
What it does is basically:
1. Patch out the check for vb->boff==0. In mmap mode, the boff field contains an offset from the base of some memory region, and for the first image buffer this offset is actually zero.
2. Replace some memory mapping code (overlay-related, I guess) with videobuf_queue_to_vaddr.
3. Fix up rk_camera_scale_crop_arm to skip some physical memory flushing (also seems to be overlay-related).
Of course this is bound to break the overlay mode, but I don't have any real use for it:)
In the rk3026 image scaling and cropping is done on the CPU, other SOCs seem to use the IPP for the job - this may require some more hacking to get the IPP to store its result in the mmap buffers.
Thank you and awesome work! Actually RK frighteningly modified much V4L2 related code in the kernel, so who knows what they've left. Great to see you got so far with it!
Delete