So ever since cell-phones came out I've been dreaming of flying remotely controlled planes located on the other side of the world.
I'm a long way from that still, but I've made a good bit of progress on some fronts.
Each idea consists of these components:
A controller which can display video and actuate controls, and transmit data over WIFI or bluetooth
A device which can transmit video and receive commands over wifi/bluetooth, and send signals to the other hardware (e.g., motor controllers) via GPIO/PWM pins
When smart phones came out, I thought, "Eh, I can make do with this, but it's not very neat... If only they supported console controllers"
When tablets came out, I thought, "Yeah, this is better than a small phone, for sure, but with the same limitations"
Then, one day I was wandering around Fry's looking for a new soldering iron, some really fine tweezers, and some card stock, and I saw this goofy damn thing on a display mount. I normally ignore everything that isn't what I'm looking for, but I saw this fucker and was like, "Shit, NVIDIA made what I wanted!"
So I bought two.
Anyway, I developed a service in python which handles UDP hole punching through the carrier firewall, transmitting MJPEG over UDP, and receiving commands via UDP, with an easy mapping interface for command->action which supports button click type actions (IE, thing is depressed, or released) -val<->+val (IE, joysticks) and 0<->val (IE, triggers), and facilities to accomodate picky things like webcams which support different features. This service can run on pretty much any linux platform with v4l-utils and python.
I also made an android app which takes care of hole punching, receives an MJPEG stream over UDP and displays it, and also wrapped the controller and made an interface to transmit controller input over UDP.
So, with the shield, a laptop, an MS LifeCam, and two cell phones with hotspot capabilities, the proof of concept is complete.
MS LifeCam, as it turns out, has a 6x or so optical zoom. Neat!
Still looking for the best mini-PC type thing. BeagleBone Black is OK, RaspBerry PI is OK, Intel Gallileo may be the winner, or who knows..
The shit is running on a laptop now, streaming decent quality 320x240 video at about 30FPS and a really small delay, I can zoom in/out with the left trigger, manually control focus with the right (helps when looking through a scope, for example), enable/disable auto_exposure which is nice because the lifecam's auto_exposure is slow as shit and will drop the framerate a lot when looking around in a room that is dark in places, and light in other places.
I just moved into a new office building and we still have service people coming in and out. I sit at one end of a long hall, and the entrance is at the other. Just saw a guy come in, zoomed in on his face, saw his glasses and stuff, zoomed out, got up and went to meet him to see wtf he needs.
Very cool.
I'll have a prototype or 3 in time for opening day of deer season. Gunna stick one a ways down the deer trail I like to hunt.
Some other ideas:
Base station to look at a target at a firing range. Nice not to have to walk all the way out to the target to see what's going on.
Remotely controlled Paintball turret
Car
Copter
Plane
I'm trying to make the suite generic enough to be applicable to pretty much anything where you want fast, good quality, first person view and the ability to do work over a celluar data network. I would have used something easier, like RTSP, but turns out that android devices have a firmware level streaming buffer defined, which forces a delay of somewhere very far North of my goal (on the shield, it's about 6 seconds!)
Great for stable streaming, shit for flying a plane. Didn't see anyone else doing anything similar with UDP, so I rolled my own. I tried it with TCP, but TCP sucks for streaming video, at least the way I was doing it. I read a white paper by some smart kid in California who was trying to make the point that with an adaptive packet filter, TCP is better than UDP, but I'm not that smart, nor do I want to spend that much time on it.
So, to reiterate the point: The Nvidia Shield is fucking awesome. Good for gaming, too.
I'm a long way from that still, but I've made a good bit of progress on some fronts.
Each idea consists of these components:
A controller which can display video and actuate controls, and transmit data over WIFI or bluetooth
A device which can transmit video and receive commands over wifi/bluetooth, and send signals to the other hardware (e.g., motor controllers) via GPIO/PWM pins
When smart phones came out, I thought, "Eh, I can make do with this, but it's not very neat... If only they supported console controllers"
When tablets came out, I thought, "Yeah, this is better than a small phone, for sure, but with the same limitations"
Then, one day I was wandering around Fry's looking for a new soldering iron, some really fine tweezers, and some card stock, and I saw this goofy damn thing on a display mount. I normally ignore everything that isn't what I'm looking for, but I saw this fucker and was like, "Shit, NVIDIA made what I wanted!"
So I bought two.
Anyway, I developed a service in python which handles UDP hole punching through the carrier firewall, transmitting MJPEG over UDP, and receiving commands via UDP, with an easy mapping interface for command->action which supports button click type actions (IE, thing is depressed, or released) -val<->+val (IE, joysticks) and 0<->val (IE, triggers), and facilities to accomodate picky things like webcams which support different features. This service can run on pretty much any linux platform with v4l-utils and python.
I also made an android app which takes care of hole punching, receives an MJPEG stream over UDP and displays it, and also wrapped the controller and made an interface to transmit controller input over UDP.
So, with the shield, a laptop, an MS LifeCam, and two cell phones with hotspot capabilities, the proof of concept is complete.
MS LifeCam, as it turns out, has a 6x or so optical zoom. Neat!
Still looking for the best mini-PC type thing. BeagleBone Black is OK, RaspBerry PI is OK, Intel Gallileo may be the winner, or who knows..
The shit is running on a laptop now, streaming decent quality 320x240 video at about 30FPS and a really small delay, I can zoom in/out with the left trigger, manually control focus with the right (helps when looking through a scope, for example), enable/disable auto_exposure which is nice because the lifecam's auto_exposure is slow as shit and will drop the framerate a lot when looking around in a room that is dark in places, and light in other places.
I just moved into a new office building and we still have service people coming in and out. I sit at one end of a long hall, and the entrance is at the other. Just saw a guy come in, zoomed in on his face, saw his glasses and stuff, zoomed out, got up and went to meet him to see wtf he needs.
Very cool.
I'll have a prototype or 3 in time for opening day of deer season. Gunna stick one a ways down the deer trail I like to hunt.
Some other ideas:
Base station to look at a target at a firing range. Nice not to have to walk all the way out to the target to see what's going on.
Remotely controlled Paintball turret
Car
Copter
Plane
I'm trying to make the suite generic enough to be applicable to pretty much anything where you want fast, good quality, first person view and the ability to do work over a celluar data network. I would have used something easier, like RTSP, but turns out that android devices have a firmware level streaming buffer defined, which forces a delay of somewhere very far North of my goal (on the shield, it's about 6 seconds!)
Great for stable streaming, shit for flying a plane. Didn't see anyone else doing anything similar with UDP, so I rolled my own. I tried it with TCP, but TCP sucks for streaming video, at least the way I was doing it. I read a white paper by some smart kid in California who was trying to make the point that with an adaptive packet filter, TCP is better than UDP, but I'm not that smart, nor do I want to spend that much time on it.
So, to reiterate the point: The Nvidia Shield is fucking awesome. Good for gaming, too.
