DIY Automobile Windshield Heads Up Display (HUD) with Kintetic and Raspberry Pi For Motion Tracking.
by john threat on October 10, 2013
So the new Xbox one has a new Kinetics sensor. Those things are awesome. I watched this video explanation here:
So this sparked an idea about mounting one on the front of the car to track the motion of cars in front of the car and feed that information to a raspberry Pi. The raspberry pi then could output the results of it’s tracking on the windshield of the car in real time with a DLP projector. Here is a quick mockup of the flow of how this project would work:
What could we do with this feed of real time data that is projected on the windshield? Imagine driving down the street and if there was a drier doing something you didn’t like – instead of road rage what you could do is launch an on-screen fireball at him and blow him up virtually! Almost like a video game, except safer since you don’t get shot. Here is an example of the windshield display:
Imagine if the Kinetics tracker could also take photos in color rapidly and the Raspberry Pi can perform Optical Character Recognition (OCR) on license plates and other objects and track them in real time. It could also potentially identify vehicles by their type – and potentially even their make and model. Certainly trucks and jeeps and certain sports cars have very unique profiles.
Theoretically it could even allow you to drive a wee bit in the dark with a pseudo-night-vision – detecting objects in your path you might not see moving – like a deer off to the side.
If the Raspberry PI had Wifi and GPS available then it could also pull some real time information on the currently GPS position from Google Local , Yelp, Wikipedia and other databases and project Arrows pointing to such local attractions. Possibly it could even spot parking spots at oblique angles before you even notice them! Let’s crack this up and see what it can do.
I think the practical application of this is quite broad, but these are my initial goals to hit. One of the challenges in this is to find a way to see the projector on the windshield in the daytime. In this regard I have done some research – and it”s going to be hard to figure out. There is special treated glass , and there are some third party spray on products, and some companies that say they have developed a proprietary way to do it with any glass. Oh well – challenge excepted!
If you have any ideas on what else can be done with this setup feel free to hit me up and let me know.
by john threat on September 25, 2013
The Big Idea:
I got a chance to ride around the Tesla which blew me away. It’s like a iPad controlled giant battery on wheels. Well, unfortunately Tesla doesn’t make a 4×4 yet. My first thought was to get a junked Tesla and rip the dashboard and transplant it into my 97′ Jeep TJ Wrangler.
Then inspiration hit.
Here is what I used
Samsung Android TAB 3 10.1 : I snarfed this from BestBuy with a extended one year all inclusive warranty just in case something went wrong.
PLX Devices Bluetooth ODB II Scanner : If you want to use Torque for real time operating data from your car, you’ll want this.
Startech Right Angle Micro USB cable : To neatly connect it to the inverter.
Right Angle 1/8″ Mini-Pin Audio Adapter: To neatly connect the audio out from the tablet to the amp/speakers in the vehicle.
Speakercraft Automatic A/B Line Switch: Optional: If you want to run the audio from the tablet over your existing speaker and you have a one line amp.
Dremel : The magic device to cut into the dash neatly . They should teach kids how to use a rotary tool in school to stimulate the imagination.
I took the Dash Bezel off and to make room in a Jeep, I had to remove the climate control – I rarely use it. That might not work for everyone – and I also lose windshield defrosting abilities. Oh well, that’s what my hand is for. I used the dremel to cut in deep notches into two existing plastic posts that hold the dash bezel in place.
I took a second to clean up the wires and clean it off from burning man. Then easy as pie, the Samsung tablet slid right in. The bezel from the Empeg/Riocar in-dash Linux player also helped keep the Samsung Tablet nice and snug.
Then I threw in the right Angle Micro USB at the bottom to prevent the stock micro USB from getting bent and destroying the power charging port. I keep a 1500 watt inverter with USB ports behind the dashboard, so AC power wasn’t far. If you don’t have an inverter, you may have to get a car charger (which I would recommend cutting and tying directly to the prongs for the cigarette lights behind the dash).
Then I plugged in the right angle audio mini-pin so it would be flush with the tablet and ran that wire to the amp. You can also run it to the auxiliary input to your head unit, but I prefer having the Google map directions to play over the music from the head unit)
The Final step was the cut the Bezel. Now I am not a Dremel expert, but the next time I cut it – I know better how to tackle it. I pretty much cut off all the cross bars haphazardly, which is fine, to be able to access the hardware buttons. Next time I’ll do it neat and then reattach the crossbars for a better stock look – or I’ll make some on the 3d Printer when I get a chance.
A side bonus is that if you press down on the plastic bezel at the top, it will trigger the power button and volume buttons!
The functionality is so amazing and a Jeep is the one car where it looks better when things are hacked together. It still looks pretty nice and stock except for close up. I placed the bezel on, and I can see the whole screen from any angle. The forward firing camera is available for Skype, Google Hangout, Vine and Photos and Videos with the native Android app.
To really enjoy this setup you will need to be able to tether the Tablet to a WIFI source. They unfortunately don’t readily sell a SIM card version – but what I did was to wire in a WIFI hotspot from my mobile carrier into the vehicle. It wasn’t what I wanted as I had to run another power cord to keep it powered, but the upside is that any device I have near I can use the internet without touching the tablet – and in a pinch – I can take the WiFi from out the dash fast for other purposes where as the tablet isn’t so easy to extract.
By the way, the forward firing camera and microphone work 100% and very well. I can do voice searches and talk to people while driving with the top down at local road speed (10-40MPH).
For the final step – I connected the PLX Devices Bluetooth ODBII scanner under the dash in the speaker well. It draws power from the ODBII port luckily. It works well, but it does leave a cool red glowing going beneath the dash and obviously this is going to be a miniature draw off the battery if left sitting forever.
I haven’t checked other peoples lists yet, but a list of awesome in-dash car apps list is probably needed. Here is a list of apps i recommend that really worked awesomely me so far:
Torque: The Ultimate car app. You can run gauges of all types while you drive for real time and historical data. It’s truly amazing for those of us who love numbers and data – and then use it to adjust our equipment. Reads your ODB II info and works with your GPS as well to do cool stuff. Skip the free version and go right to the paid version – you’ll definitely want it.
Google Now: This is the new tool that pretty much acts like an amazing personal assistant. I love it. It calculates how long it’ll take to get home and to work, track airplane flights, yours and people coming to see you, and packages for you, (you can see my Carhartt Long sleeve shirt is on it’s way) track your meetings and more. It’s still evolving but I am already hooked and keep it up on the front screen and check the full screen version often.
Google Maps: Big Daddy! The best map and turn by turn navigation on the map with satellite terrain – and it works all over the place – from the streets of L.A. to deep into Death Valley – and it remembers your favorite and recent searches for quick returns to places. Drop a pin in a place you want to explore and it will get you there. The downsides is that is should be easier to save a place permanently and you should be able to swap what side the navigation arrow goes on.
My Tracks: Awesome app that tracks a line on Google map for you – so even if you lose a signal you can find your way back. You can also share the track on social networks or just keep a running tab where you have been yourself by easily dropping pins. It includes all types of metadata like speed and altitude on your trip as well. Then you can share your track or save it for later!
Gmail: Short description: Awesome email client and easy to read and navigate on the tablet. Beautiful and now if someone is taking too long to come back to the car, who cares! I’m doing business! Of course what is nice is addresses and contact info and other links Google processes open the corresponding apps nicely (maps, calendar, etc)
Tablet Talk: Awesome app that lets the text messages from your phone appear on the tablet. You can also send texts back to them from the Dashboard now! It’s also shows if someone calls and you can take the call as well! This is really awesome tooling around town or hooking up with friends, or picking someone up from the airport by being able to see the text as you are driving. You can set it to reconnect automatically as soon as you get in the car.
GoPro: Come on – connect to exterior mounted gopros? Awesome. Backup camera? Record crazy stuff at will without getting out of the car? Change settings on the fly from the car on all the cameras? A Must have for the Go Pro Hero 2 and 3. I’m totally in love with this – the only issue now is that you have to jump onto the cameras wifi , which means no internet- awww.
Zello: This is a walkie talkie push-to-talk application. You have to make an account, but it allows you to sort of radio to your car without making a phone call or text. You’ll still need a signal . but it allows me to pretend my Jeep can talk when I am not standing near it. It scares the stuffing out of people. It’s also good for co-ordinating movements with people with picking them up without worrying about making a phone call or holding a phone up while driving. I’m into this.
Skype: Oh yeah, now you can skype (or google hangout) with people while driving or parked. I recommend parked – but it def kinda awesome to connect to people from the drivers seat to catch up or do a quick conference call for work.
DS Video and Other Synology Apps: If you have a Synology NAS these apps let you connect with your home server to pull up media and documents. Stream movies and music to your dashboard from home, or maybe you need that PDF of a report to mail to someone? Just to turn it up, there is DS Download which lets you add torrents in the field, so basically – well you can watch it when you get home.
Pandora: I know some people like other services, but I find i discover the most amazing arcane records on Pandora. One issue is that their navigation on android tablets is t-e-r-r-i-b-l-e . The plus side is I am considering a paid subscription because getting video ads while driving and blasting music is jarring and eating up bandwidth.
Yelp: Come on, Yelp has made itself indispensable to quickly find things like Jamba Juices or a good sushi spot. It does quite a bit more – and using Monocole on the dash works better than when walking.
Voice Search: Yup! Works well – both google and samsung voice search. I lean toward using google voice search more just because those are the apps I am using the most.
Camera: Come on, taking action shots and videos of all your friends while driving and having fun is awesome. You can even voice trigger the photos if you like, by saying things like Cheese! , Play with the settings, there are plenty. The default camera app is really well thought out and mega fun from the drivers seat! A new dimension in selfies.
Phone Fusion Visual Voicemail Plus: I favor this – for some reason – everyone kept me away from Google Voice. Well I use this to visually see a list of my voice mails and play them from the dashboard. I can forward them via email and save the .wav file to stop you from suing me later. You can have them transcribed for a price too – which I am considering paying for.
Pulse: My personal preference for a visual style newsreader. Easy to add news sources. Awesome in every way and as widgets for easy reading on the home screen. Highly recommended
Music Visualizer: Awesome large screen visualization, works with music players on the tablet, as well as taking input from the microphone.
Challenges and Errors:
Heat: Right now I’m driving thru the desert a lot, and it looks like it survived sustained 120+ degrees heat multiple times. It did give me warnings galore about temperature and charging. I realize heat can kill or degrade the systems in the tablet, and its taking massive heat from the firewall and whatever still seeps thru the air ducts. I’ll have to figure that one out. It doesn’t complain till its over 110 degrees in the air and the jeep is running for a while.
Security: Come on, this is probably begging for someone to steal it, at the same time – dash board screens are so prevalent, it doesn’t really look special from a distance. I turn the screen off when I leave. What I’d like to do is get a 3D plastic print of a black door to cover the screen when I leave the car unattended for a long time in a public place. Also – if anyone has ideas how I can lock it to the dash more securely , please send your suggestions. So far though – so good!
App configuration: Some apps don’t play well in forced landscape. Very few but they are out there. Some have terrible navigation , like Pandora. Some apps still could use more configuration options (like it would be nice to swap what side the turn directions go in Google Maps/Navigation for instance)
Danger: Lets face it, with all this stuff at your finger tips – you might be tempted as a driver to use stuff other than maps or music visualization while driving. So here is a disclaimer – of course if you choose to do this, this is of your own accord. You have to pay attention to the road when driving. Flashing your eyes at a incoming email or text title on the screen at a long red light is one thing. Engaging in reading and writing emails and text at 60mph or more is dangerous and irresponsible.
Brown Color: The Samsung Tab if you go that route only comes in White and Brown. I chose Brown. I wanted black – color me disappoint. I am trying to weigh my options for covering the brown bezel on it with black without covering the camera. You don’t notice it at night but in sunlight, it’s brown. I’ve forgotten it’s brown over time.
Start From USB: I am already on push button start, but what If I could start the car from the USB port? Or from the Wifi network? Totally doable. I have to get on it.
Well let me know what you think — I’ll answer your email as soon as I park.
by john threat on September 25, 2013
Oh Yeah! So we finished up Monsters. It’s a show about two horrible Monsters , Dick and Dash , that are roommates that fight, curse, drink,
fuck, take drugs and smoke – in other words it is a story about heartfelt friendship. We rocked out three awesome pilot episodes with a raft of talented actors (Daniel, Marc, Bettina, Al, Mina and more – thank you!). We then had a quick screening among friends to see what people thought:
Oh shit! They loved it! Now what. It has touches of The Mighty Boosh, The Young Ones, The Odd Couple, Pee Wee’s Playhouse, Adventure Time and yet it is it own thing you’ve never seen before. Check out a few still below from the show:
Okay – so I have an idea. I packed up my Jeep Nomad with an insane PDF 24 episode guide and bible of the show along with 3 pilots that are nose bleed intense and I’ve headed toward L.A.: big city of dreams to kick it with like minded people who like to make heart rending touching pieces of comedy with a deep felt humanity that just happen to have an occasional piece of shit on the screen. Is that so wrong?
So if you are in L.A. and you wanna toss back a high IPA beer or 12 year old whiskey in Silver Lake or Echo Park, let me know.
by john threat on September 20, 2013
I rushed head first into the 3D printing revolution – it’s the stuff dreams are made of. One thing that bothered me was the smell and airborne residue of burning plastic in a small New York apartment – but with the current generation of 3d printing you still need to keep an eye on the build process. So I came up with the idea of converting a utility closet into 3D printer closet! It’s really attractive and works well – now I don’t have to smell anything and worry about breathing in fumes.
It’s pretty simple. Take the door down , draw a level square on it and then cut it out.
I grabbed some plexiglass from home depot and glued it on the other side. Some tape held it in place to make sure it didn’t fall out before the glue dried.
Take this time to put a cooler handle on it. I really wanted to put a hi-tech LED handle on it but I haven’t found one yet. (Keeping an eye on this one!) – So I threw a fancy stainless steel handle on it that looks awesome – but it isn’t what I really want.
This project only took about a two hours including a drive to Home Depot for the plexiglass, glue, wood, spray paint and tape. I have a Makerbot Replicator 2 which i bought just because of looks, but its not really my first choice for a 3D printer. One of the things that sucks is that it isnt internet aware, which means i have to bring a SD card over to it. Too bad it doesnt have WIFI or ethernet – all the best printers too. I screwed myself on that one. Oh well – get up lazy. (WIFI STILL BETTER)
I also installed HUE lights all over my apartment on a suggestion from highly recommended entertainment and business lawyer Quinn, which makes the lighting look even more amazing. With all the stuff you can do with HUE lights from Phillips – I’ll have to write something up on that separately that shows off what it can do when hacked into everything in the place. (if only the 3d printer had WiFi then it could totally alert me on a finished print by blinking all the lights!)
Sorry my apartment is such as wreck – but I didn’t fix it yet from finishing the shoot on Monsters. All in all it came out awesome as functional eyecandy!
by john threat on May 24, 2013
So rumbling in the pipeline is a video for MNDR: Faster Horses.
MNDR is a talented artist who I was double super lucky , no like mega lucky to work with. We met and tossed up some whiskey and she introduced me to Guy Maddin: Sissy Boy Slap Party
Man! That is genius, no? I love that short film – it’s like in me top ten now. I love what she and Peter Wade did on the “Feed Me Diamonds” album and I was pumping in the Jeep Off Road and i swear I wrote like five treatments – the one she picked was also my favorite – Daymare! Inspired by the lyrics and music I took all the girl’s dreams I’ve ever heard and transposed them on a side scrolling video game thematic. She literally travels thru different spheres of the dream state to reach the inner sanctum sanctorum to free herself! Its a frame one to frame end adventure and it’s be out faster than you can shake a stick. Here are a few photos:
by john threat on April 24, 2013
Sterogum the awesome music site debuted the MNDR/Faster Horses music video I directed. Spin also had a great write up!
by john threat on March 4, 2013
DayWalker attempts to literally record your whole day and condense it into a music video at night and upload it to YouTube. Imagine smashing your ennui with your mundane existence enhanced with soundtrack – and instant Kafka-esque Die Verwandlung of your journey through this world. DayWalker explores your environment both online and off and mashes it up into a short film that you can save for posterity or share with friends. You can even analyze it for connections and patterns you didn’t even know existed, like Albert Brooks in “Defending Your Life“.
So things are progressing rapidly thanks to the Raspberry Pi! Let’s take a look at the hardware spec first:
So DayWalker hardware consists of a Raspberry Pi as it’s brain – outfitted with a shiteload of things to interact with the environment around your. GPS – to grab your position on the planet. Bluetooth and WiFi to scan all devices and networks that are near you. A microphone to catch and analyze all the audio around you and break it down into words and relate that to your day as well. Let’s wrap the package up nicely with a touch screen that let’s you know the status of the device, and on-off switch and a rechargeable lithium-ion battery all wrapped up in a custom 3D printed case with two arms straps.
Now let’s drill down into the meaty software spec:
WTF? Okay the short of it is this unwieldy Plesiosaurs of a diagram. The right side of the diagram represents the real-time data collection from the sensory devices attached to the Raspberry Pi which is stored in the DayWalker‘s Database. This includes WiFi network names, Bluetooth phone names – possibly phone calls from your own phone, music playing near the device, conversations that you have and of course the very section on earth you are standing on.
At the end of a set time period (midnight!) , or at any time the user of DayWalker can trigger it to generate a video of what it has collected. Then the Middle of the diagram kicks in collecting additional data from the last period from various social media sources of the user : I.E: Instagram, Twitter, Google calendar, Facebook, Flickr, and other social networks to generate the video.
All this goes to the render engine that begins to pull photos for all the processed audio it has heard during the day, and match photos and positions to WiFi networks, map locations, calendar events, etc. It then beat matches a nice piece of music, or several pieces depending on the length of the video. After compiling it – it uploads it to a youtube account of your choice!
Additional software processes run a web interface for real -time monitoring and input and output to the touch screen for setup and real time updates.
Now that the idea has been laid out clearly – Let’s shake the caboose and see what comes loose.
by john threat on February 14, 2013
Everybody Watching Me is an experimental automated low-fi mashup engine in the pursuit of the soi-disant hacker-art.
Everybody Watching Me takes live feeds from open web cameras around the earth in real time and mixes it with cool audio tracks to effectively create a running 24/7 days a week music video out of the mundane minutiae of people’s lives to satisfy the techno-voyeur in this global self-surveillance society.
I tweeted out the idea, and then i thought, why not flip the phodeo engine that powers TweetCinema to do this. So I set to the task of coding this up. That’s the high concept, let’s hold the bolts and look at the nuts of this operation:
Here is a flow chart of the original concept before vi ever opened a text file on this project:
The engine grabs 30 frames from each camera around the world, filters them by several given criteria (The most crucial one is timezone which is coded in but not turned on yet) and then assembles them into a music video with random music tracks & uploads it to the Everybody Watching Me Youtube Channel.
The engine also pushes out a realtime live stream from the website that you can view – that can run pretty good if you have a fast connection – and you can also watch it on video delay which is a loopback archive in mp4 format that is easier to digest across platforms.
I cobbled together an extra function to do some light motion tracking. Right now I kept it to the first two frames, but it could easily be let loose to track all frames. It’s a simple threshold compare to find out where there is motion. Here is a example below. The top box is a TV monitor, the bottom box is a persons arm that is moving, cleaning a table:
Once if finds motion, at the moment, it selects only one, and the biggest motion it can find and attaches a SMS style text speak to who or what is ever moving at their loci, effectively hypertagging them in real time. The text is animated just for even more music video immersion. Below is an example of a lady reading a paper the engine tagged with WTF!
I think clearly the time zone bit needs to be activated to avoid looking at cameras that are pitch dark because it’s nighttime. I also dont know yet if 30 frames is the sweet spot. It currently takes 2 or 3 hours to run a circuit of cameras and right now the database is polling only 160 cameras out of maybe 3-4 thousand available.
Faster server? Less Frames? Code tweaks? When pulling data from foreign websites you are sometimes at the mercy of the uplink and the timeout. A faster circuit run would really be awesome – any ideas on that are welcome.
I already rewrote the motion tracker to handle multiple objects with a tighter center target. I think it would be cool if it could put animal heads on people who are moving or little cute icons. I do plan to cut on the full tracker across all 30 frames and let the words follow them where they go, that seems like that would be awesome.
I think maybe a cute little flag of what country the camera is at might be a nice ideas as well. I also think a frame rate pull based on the beat of the music would also lead to cuts that land on the beat of the music.
Another idea is that instead of a linear poll, perhaps it should poll cameras randomly. Another idea, is of course threading the engine for multiple pulls which would speed it up considerably.
Yet another idea would be to analyze the words in songs and pull camera footage that the engine thinks matches the lyrics, or conversely switch the music based on the images!
Finally I think I’ll add in a end slug with the logo identity for the project – just for that professional touch! everybody watching me – directed by The World!
Let me know what you think, and don’t forget, someone is always, always, always, watching….
by john threat on February 5, 2013
What it does simply is take the top trends off twitter and match them to photos using a simple multi-stage algorithmic process to find the right photo from primarily public domain images. It’ll eat any kinda of photo as well and it stores it in a database for later future projects (big data makes apps smarter!)
Now instead of trying to figure out what that cryptic hashtag that everyone is talking about – you can quickly see who, what, where and occasionally how and why people are talking about that subject on twitter! I find it useful myself from where an geopolitical events are taking place – to sports players names who are being traded, to celebrities pulled over for DWI and actually a lot more – all in real time. Big Data, Game Theory, Predictive AI in baby steps.
So the extra special touch of a hybrid programmer/director – every day at night the server also automatically generates a music video out of the top trends of the day!
It assembles it all into a dupstep track that even says the name of the trends and then uploads it to YouTube with a description and hashtags. Here is an example of a completed one:
The idea is to generate one daily, monthly and a year in special all automatically from the server. Set it and forget it. The whole channel is here @: [ tweetcinematrends ]
I’m gonna extend these auto-generate videos a bit more with an extra special live component. I just have to cast for the perfect humans. More on that shortly – but will come to the table in the next 30.
I also have a bit of SEO magic going on the web interface. Each trend that enters the grasp of tweetcinema it generates a static page and directory with the image associated with it. A an example is a trend #BongoTakeover that ranked today. The server generates this: http://www.tweetcinema.com/trends/Bongo_Takeover/ which is all SEO friendly in everyway. The only thing holding it back at the moment is that they aren’t technically linked anywhere to enter the zeitgeist. That’ll change shortly.
I found out while just running tests, things were getting weirdly meta! TweetCinema tagged photos were winding up high up in search engine image results since they were tagged properly. Even stills from the YouTube videos were ending up in searches. Not to mention YouTube video SEO is one of the least exploited high priority search results on the whole internet. Yet another dimension to the experiment.
Right now it’s international focused, which means certain times of the day it can be dominated by various regions and languages. I find it interesting that the primary movers on top international trends tends to be USA, Latin America and Turkey. Every once in a while South Korea, France, England and Russia sneaks in.
The next steps for TweetCinema is to enable on the web interface for weekly, monthly and yearly buttons so one can review longer tail trending information – and then of course a companion mobile app for Android and iOS so you can take it on the go. Maybe – and just maybe – a XBMC version too! (What about a Hewlett Packard Printer app? is that too far?)
I’m finishing up post on Monsters so hopefully I can coded both those apps in the next 30 days. If you have any questions just ask!
by john threat on January 23, 2013
So I’ve got these ideas of bridging AI, Game theory and Data Visualization into a new level of awareness in-betwixt directing for a living. It sounds grand on the macro scale but let’s start small. With the advent of the internet all the data we longed to shove into a computer for processing is here. This will have profound effects on game theory, AI, visual recognition (there goes privacy!). A lot of researchers see it. The singularity is near – but I’m getting ahead of myself.
Enter Tweetcinema: The idea is simple. Take the top ten trends on twitter and match them to relevant images. The view of tweetcinema will no longer be faces with a cryptic hash tag, but rather an image to quickly absorb in 10 frames what the zeitgeist is talking about. Doesn’t matter if it’s Justin Beiber or an earthquake in the pacific ocean – tweetcinema will show it to you quickly and you can digest it in seconds without having to cut and paste a hash-tag term. Here is a flow chart of the scope of the project:
If you are a decision-maker – especially let’s say on youth entertainment – you could scan this and readily find out whats going on and what kids are talking about instead of waiting for your intern to decipher hash tags and bring it to you. If you tuned into global events you could rapidly see what is developing in a foreign country even if you don’t have the names and places down, the images would effectively give you a leg up on processing this.
The key is developing an algorithm that sorts metadata on the net from various image resources and match the right kind of photo. The upshot is that this also helps in turn categorize more photos with metadata that can be used in other projects across a wide scope.
I actually had this idea two years ago and started coding it – but did some directing – but back to coding! I plan to use the engine for two or three other ideas rattling across my notebook like Daywalker and Atlas Eleven. More soon!