Synchronized HTML5 Video with GPS Data
YouTube is littered with first person helmet camera videos of every kind. In fact a quick search turned up approximately 800,000 videos. They’re great at showing some high-intensity action up close and personal. The problem with helmet cam videos is that they lack the ability to convey any information about the bigger picture. Where exactly was the mountain that the cliff jumper was diving off of in that awesome video I just watched? How fast was he going when he deployed his parachute? Where did he land, and how large was the landing area? These are really interesting questions, but there’s generally no way to get this kind of information from the video.
With this in mind, I set out to do something a little different. Armed with a GoPro helmet camera and a GPS tracking app (Google My Tracks) for my Android phone, a friend and I headed out to the Rocky Mountains in Utah for some alpine skiing fun. We recorded lots of video and about 25 hours of GPS tracks. With all this data and Cesium, I had everything I needed to build a simple app that combined the micro perspective captured with the helmet camera with the macro perspective that comes from GPS information. I called the project Powder Tracks.
Cesium is used to show the skier’s track on a 3D globe with 10m National Elevation Dataset terrain and high-resolution Bing imagery. Videos taken during the trip are synchronized with the skier’s position and play along side of the animating visualization. Billboards are used to indicate the skier’s current position and the location of available videos. Camera flights provide smooth transitions to the next location when a user selects a new video to watch.
One of the great features in WebGL is the ability to interact with other web technologies, such as HTML5 video elements. By passing a loaded HTML5 video element as the source in a call to
texSubImage2D, the current video frame is copied into a WebGL texture. We used this feature to create a
DynamicVideoMaterial that can be used to texture various Cesium and CZML objects including polygons, ellipsoids, polylines, etc. through the Cesium Material system. With a little bit of timing logic, the video texture is kept in sync with the animating scene. The DynamicVideoMaterial is being developed in the Video Branch.
High Resolution Terrain
What good is a 3D scene of mountains if you don’t have accurate terrain information? Out of the box, Cesium has the ability to render streaming terrain information via the CesiumTerrainProvider. By using some Geospatial Data Abstraction Library (GDAL) tools, we converted high resolution terrain data from the National Elevation Dataset for use in Cesium. The data used was 1/3 arc second (~10 meters) elevation data covering the state of Utah. To generate the elevation tiles, we first used
gdal_translate to convert the raster data to GeoTiff format and then used
gdal2tiles to generate Tile Map Service compatible elevation tiles.
After working with the prototype application for a while, I realized that even the high resolution terrain didn’t easily convey all of the information that I wanted. Particularly the ability to easily identify very steep areas on the mountain. The solution to this was achieved by again using the NED elevation data. This time, instead of converting the data to elevation tiles, I converted the data to imagery tiles where pixel colors corresponded to the slope of the terrain. The new tiles were loaded into Cesium using a TileMapServiceImageryProvider and were rendered over the Bing imagery. To perform the conversion, I again used GDAL tools. First
gdaldem slope was used to convert our elevation GeoTiff to a slope GeoTiff where each pixel value represents the slope (in degrees) of the terrain. A second call to
gdaldem color-relief converted the slope GeoTiff to a slope-shade GeoTiff by mapping slope values to my user-defined colors. Finally, I used
gdal2tiles again to generate a TMS tile pyramid.
One of the powerful features of Cesium is the ability to load and visualize time-dynamic data via CZML. I used various portions of the CZML Writer project to convert my data into CZML. Converting the GPS track data was the easiest part. The GPS app I used to collect the data saved all of the information in KML format using the Google gx extension. Using the
CesiumLanguageConverter all of my GPS tracks were automatically converted to CZML
Paths. Slightly more troublesome was generating CZML
Markers that would indicate the georeferenced locations of where videos were captured. I could have manually searched through the GPS data to match the video time tags with GPS positions, but I had about 60 videos and days worth of GPS data. That process would have been extremely painful. Instead, I bulk loaded the video times and the GPS positions into a Microsoft Access database. By using a simple C# application I could then easily query the database to find the locations of all the videos. With this information available, I was able to use the C#
czml-writer library directly to generate CZML for all of my video markers.
With the GPS track data loaded into PowderTracks I had the ability to extract additional “realtime” information about the skier. All of the data that I loaded via a CZML
So head on over to the Demos page to run Powder Tracks for yourself. I’d love to hear your ideas for new features.
- Cesium Version B23 Released
02 December 2013
- Best of the Cesium Blog - Year One
13 November 2013
- Geometry And Appearances
04 November 2013
- Cesium Version B22 Released
01 November 2013
- Apps4Philly Hackathon
22 October 2013
- GitHub Pull Request Workflows
08 October 2013
- Cesium Version B21 Released
01 October 2013
- Cygnus Mission Demo
18 September 2013
- Cesium Version B20 Released
03 September 2013
- ESA SOCIS 2013 Students Announced
27 August 2013