Realtime GeoJSON Data Stream Access and Changelog

Find access information at the bottom of this page.

Status / Changelog

04-27-2016 14:00 PT

Socket.IO connections now require Socket.io 2 or higher.

04-27-2016 14:00 PT

NTripCaster became mostly jammed up again today, I restarted it around 13:33PT.

04-25-2016 12:15 PT

NTripCaster became mostly jammed up again around doy 115. I restarted it today and seems fine.

04-08-2016 09:14 PT

This morning's problem is RTNET cannot contact any of the PBO stations used for for RTNET corrections server. Will notify group...

04-08-2016 08:55 PT

RTES not outputting data, looks likely it stopped outputting around 00:30PT. Debug continues...

02-15-2016 10:10 PT

Power outage to (at least) our router at USGS last night and this morning approx 23:49 - 12:21 PT. RTES and RTNet processes did not survive. I restarted RTES, RTNETs, and rt_dump processes and streams are flowing again.

02-05-2016 19:52 PT

RTES machine stopped reporting at about 12:28PT today. After machine began responding again at about 12:44PT, RTES remained in a deadlocked state. Have restarted RTES and RTNETJSON streams are flowing again.

01-23-2016 11:38 PT

Some rt_dumps were spinning out of control, intermittently bogging down RabbitMQ. Fixed now, streams flowing.

01-19-2016 11:45 PT

Some rt_dumps began spinning out of control last night or this morning, jamming RabbitMQ. Fixed now, streams flowing.

01-11-2016 12:19 PT

A number of rt_dump processes that were restarted this morning caused RabbitMQ to jam. Fixed now, streams flowing.

12-29-2015 09:30 PT

RTES crashed again around 01:30PT. Restarted RTES, RTNET processes, and associated rt_dump services, streams flowing again at 9:30PT. Will contact GPS Solutions, this has happened numerous times now in recent weeks.

12-22-2015 05:06 UTC

Report of RTNETJSON streams stopping around 1700PT. RTES process still running, but cpu usage fell off at 1700PT. Restarted RTES and verified streams flowing again at 12-22-2015 21:06PT.

12-15-2015 22:05 UTC

Restarted GSOF streams and data flowing again, were intermittent/extremely slow, coming through in chunks, prior to this. RabbitMQ seems to bog down when a stream service starts to spin, or misbehave in some other manner.

12-14-2015 16:45 UTC

Restarted RTES, RTNETJSON streams flowing again. RTES appears to have crashed around 12-13-2015 06:30PT.

12-08-2015 18:02 UTC

Restarted RTES, RTNETJSON streams flowing again. RTES crashed around 14:27UTC, although streams seem to have actually stopped for a currently unknown reason around 06:00UTC.

11-11-2015 01:52 UTC

RTNET streams back up. Problem was our automated download of sp3 orbits was broken.

11-09-2015 20:10 UTC

RTNET corrections server appears not to be working. 5 of 17 PBO sites used are down, possibly related. Continue to debug...

08-17-2015 03:03 UTC

gpsrt5 replaces gpsrt2. Binex is being written to disk and streams are being passed through to RTES and rt_client. However numerous issues remain to be cleaned up in coming days (incl NtripCaster webpage display is down).

08-16-2015 20:03 UTC

Around UTC 2015-08-16 15:35, a drive failure brought down gpsrt2, the machine accepting our realtime streams. A replacement VM is being built...

07-30-2015 20:03 UTC

At UTC 2015-07-30T16:22:04, RTES lost connection to all PBO sites used for corrections, and all RTNET streams are down as a result.

05-20-2015 ~00:13 UTC

Restarted RTES, now using BINEX instead of RTCM3 for source data, and req_nav_records set to YES
Restarted RTNet processes and correction server, now pointing at versioned CRD files, and USE_CODE_TYPES turned off.

04-03-2015 ~23:50 UTC

Restarted RTNET server with MAXTWAIT 1 "2", MAXTDELAY 1 "5".

04-02-2015 ~22:35 UTC

Restarted RTNET server -- removed 4 stations with frequent >1m latency: P159, P345, P735, P154.
Restarted all RTNET clients with shorter allowed wait times: MAXTWAIT 1 "10", MAXTDELAY 1 "5", INP_SSR_MAXWAITTIME 1 "7".

03-13-2015 ~00:23 UTC

Restarted RTNET server to provide 1hz corrections, had been providing at 5sec. Restarted all RTNET clients, unchanged at 1hz. Restarted all rt_dumps associated with RTNET streams.

02-19-2015 ~22:00 UTC

Restarted all streams. Quality indicator no longer harded to 0, but based on PDOP: (10 - PDOP) * 10, rounded to 4 decimal places.

12-17-2014 03:45 UTC

All RTNet processes restarted using new RTNet version with memory leak fixes: gpss_dist_current_CentOS-5.9-x86_64-2014-12-16.

12-15-2014 ~06:00 UTC

RabbitMQ became unresponsive and had to be restarted.

12-03-2014 00:30 UTC

All streams are back. Original RTES/RTNet problem remains unknown. RabbitMQ became unresponsive when streams returned and had to be killed. Took the opportunity to run updates (including new version of rabbitmq) and reboot machine.

12-03-2014

All RTNETJSON GeoJSON streams currently down. We're troubleshooting the issue.

11-21-2014 00:20 UTC

Restarted all RTNET rt_dump processes to fix sampleRate. With previous change to 1hz, pseudo SNCL bandCode was adjusted, but not sampleRate.

11-20-2014 04:00 UTC

RabbitMQ restart, too many unack'd msgs in Results2Earthworm, file descriptors and socket descriptors nearly all used.

11-17-2014 23:37 UTC

Assigned RTNet machine with more CPU and RAM, and restarted. Applied new license.

11-17-2014 ~23:00 UTC

Updated itrf_apr, and station_info and then used rt_update to update client coordinate file (USGS_ALL_USE.CRD) and restart stations in RTES.
Restarted all RTNet processes.
Waiting to hear from GPS Solutions re: a new license for again assigning more CPU and RAM to RTNet VM.

11-10-2014

All PPPAR rt_dump processes were moved to a dedicated machine.

11-08-2014 00:08 UTC

Restarted all RTNet streams at 1hz. Significant (~10min) and increasing latency out the frontend (rt_client) until killing some high load rt_dump processes, after which front end pppar now ~7s behind current time. Plan to migrate pppar rt_dump processes to their own machine.

11-07-2014 21:40 UTC

A lot downtime over the past few days as we tested and moved to a new version of RTNet (gpss_dist_10635_CentOS-5.9-x86_64-2014-09-02).
We are now generating our our solutions (the same PBO subnet as before) with RTNet.
A new rt_client.js attached to this page, can now provide a second arg "local" or "utc" to convert epoch time.

10-29-2014 17:55 UTC

RabbitMQ 3.3.4-1 crashed and was unable to restart (unable to allocate heap space) around 09:00 UTC.
VM was swapping a lot last 2 days. Reallocated -- doubled memory and CPU to 32gb and 8.
Upgraded RabbitMQ to 3.4.0-1 and restarted realtime stack at 17:35 UTC.

10-24-2014 18:06 UTC

Restarted all streams for change to GeoJSON v0.16.

Realtime GPS GeoJSON v0.16
- Added sampleRate field.


10-23-2014 20:44 UTC

A number of RTES, RTNet, rt_dump restarts last night and today to move to new RTES.
RTES version now in use: gpss_dist_10635_CentOS-5.9-x86_64-2014-09-02
Adjusted GeoJSON format.
Old RTNet still in use, waiting for license.

Realtime GPS GeoJSON v0.15
- Removed UTC field.
- Added time field, which is ms since epoch


10-14-2014 18:29 UTC

Restarted all streams for update to v0.14.

Realtime GPS GeoJSON v0.14
- Add temporarily-hardcoded sampling rate in SNCL field
- Add quality field, a range of 0-100, temporarily hardcoded to 0.
- Remove units. Units and metadata will be now maintained on the web


10-08-2014 21:12 UTC

Restarted all streams for update to v0.13.

Realtime GPS GeoJSON v0.13
- Trim decimal digits on lat, lon from 10 to 9 and meters from 5 to 4


09-30-2014 21:08 UTC

Restarted all RTNETJSON streams for update to v0.12.

Realtime GPS GeoJSON v0.12
- RTNETJSON stream - bug-fix: In previous versions UTC was GPS time (UTC + 16 seconds). Fixed - now subtracting leap seconds.
- RTNETJSON stream - explicitly display centisecond precision for UTC timestamps (in preparation future sampling rate increase)
- RTNETJSON stream - add (pseudo) SNCL field
- TrimbleJSON stream - add (pseudo) SNCL field


09-26-2014 20:19 UTC

Restarted all TrimbleJSON streams for update to v0.11.

Realtime GPS GeoJSON v0.11
- TrimbleJSON stream bug-fix: In previous versions UTC was GPS time (UTC + 16 seconds). Fixed - now subtracting leap seconds.


09-26-2014 18:19 UTC

Restarted all streams for update to v0.1.
Updated plotting, clear your browser's cache

Realtime GPS GeoJSON v0.1
- include measurement units
- remove GPS time and epoch seconds. Now a single UTC timestamp per record
- remove excessive precision: decimal degrees now have 10 decimal places, meters 5 decimal places




Example GeoJSON v0.16 Records

RTNet

{
   "features" : [
      {
         "geometry" : {
            "coordinates" : [
               -2458218.6965,    // meters.
               -4680467.4375,    // meters.
               3556758.6496    // meters.
            ],
            "type" : "Point"
         },
         "type" : "Feature",
         "properties" : {
            "YError" : 0.0403,    // meters.
            "coordinateType" : "XYZ",
            "ZError" : 0.0504,    // meters.
            "XError" : 0.02,    // meters.
            "positionType" : "processed"
         }
      },
      {
         "geometry" : {
            "coordinates" : [
               -0.1837,   // meters. x position (a-priori + estimate) - reference x
               -0.1341,   // meters. y position (a-priori + estimate) - reference y
               0.0182   // meters. z position (a-priori + estimate) - reference z
            ],
            "type" : "Point"
         },
         "type" : "Feature",
         "properties" : {
            "coordinateType" : "XYZ",
            "positionType" : "delta"
         }
      },
      {
         "geometry" : {
            "coordinates" : [
               -0.0994,   // meters. north position
               -0.1003,   // meters. east position
               0.1793   // meters. height position
            ],
            "type" : "Point"
         },
         "type" : "Feature",
         "properties" : {
            "coordinateType" : "NEU",
            "UError" : 0.0588,    // meters. height position error
            "NError" : 0.0271,    // meters. north position error
            "positionType" : "processed",
            "EError" : 0.0194    // meters. east position error
         }
      }
   ],
   "type" : "FeatureCollection",
   "properties" : {
      "topic" : "RTNETJSON.pppar.SolPhase_L3_se.CLAR",
      "station" : "CLAR",
      "time" : 1414097239000,    // epoch time ms
      "SNCL" : "CLAR.CI.VY_.20",    // * pseudo SEED SNCL
      "theType" : "SolPhase_L3_se",
      "sampleRate" : 0.2,
      "quality" : 0    // quality indicator, 0-100. (10 - PDOP) * 10
   }
}


Trimble GSOF RTX

{
   "features" : [
      {
         "geometry" : {
            "coordinates" : [
               -117.093195814,    // decimal degrees. longitude
               34.116408269,    // decimal degrees. latitude
               762.1656    // meters. height
            ],
            "type" : "Point"
         },
         "type" : "Feature",
         "properties" : {
            "coordinateType" : "LonLatHeight",
            "positionType" : "processed"
         }
      },
      {
         "geometry" : {
            "coordinates" : [
               -2407751.1894,    // meters. Earth-Centered, Earth-Fixed X coordinate
               -4706536.5556,    // meters. Earth-Centered, Earth-Fixed Y coordinate
               3557571.5711    // meters. Earth-Centered, Earth-Fixed Z coordinate
            ],
            "type" : "Point"
         },
         "type" : "Feature",
         "properties" : {
            "coordinateType" : "XYZ",
            "positionType" : "processed"
         }
      },
      {
         "geometry" : {
            "coordinates" : [
               0.2721,    // meters. Earth-Centered, Earth-Fixed X delta between rover and base position
               0.3916,    // meters. Earth-Centered, Earth-Fixed Y delta between rover and base position
               -0.228    // meters. Earth-Centered, Earth-Fixed Z delta between rover and base position
            ],
            "type" : "Point"
         },
         "type" : "Feature",
         "properties" : {
            "coordinateType" : "XYZ",
            "positionType" : "delta"
         }
      },
      {
         "geometry" : {
            "coordinates" : [
               0,    //** Currently invalid
               0,    //** Currently invalid 
               0    //** Currently invalid 
            ],
            "type" : "Point"
         },
         "type" : "Feature",
         "properties" : {
            "coordinateType" : "NEU",
            "positionType" : "delta"
         }
      }
   ],
   "type" : "FeatureCollection",
   "properties" : {
      "ENCovar" : 8.63989953359123e-06,    // dimensionless. Covariance east-north
      "topic" : "TrimbleJSON.pppar.RTX.7ODM",
      "station" : "7ODM",
      "NError" : 0.0151,    // meters. Sigma North. 
      "time" :     1414098329000    // epoch time ms,
      "UError" : 0.0435,    // meters. Sigma Up. 
      "SNCL" : "7ODM.CI.LY_.10",    // * pseudo SEED SNCL
      "theType" : "RTX",
      "quality" : 0,   // quality indicator, 0-100. (10 - PDOP) * 10
      "sampleRate" : 1,
      "EError" : 0.0115    // meters. Sigma East. 
   }
}
* pseudo SEED SNCL. Band code temporarily hardcoded. Coordinates given as a triplet, so orientation is hardcoded to "_". Location Code = Processing Package Code + Processing Type. Processing Package Code 1 = Trimble RTX, 2 = RTNet. Processing Type 0 = PPPAR.

** NEU deltas currently invalid. North, East, and Up deltas of the vector from the base to the rover (in meters) projected onto a plane tangent to the WGS-84 ellipsoid at the base receiver


Access

Formatted into GeoJSON, the output for the real time data then runs through our RabbitMQ server and is available through our NodeJS powered SocketIO instance.

The simplest case to access the GeoJSON streams directly is to do the following:

  1. Install NodeJS
  2. Install latest version of Socket.io-client (2+); sudo npm install socket.io-client
  3. Download the rt_client.txt script attached to this post and rename to rt_client.js

You will need to copy the script into the same directory as the node_modules directory created by installing the Socket.IO-client package in order for the libraries to be found.

Running the script without argument will attach to our server and stream all of the output data for every site. You can limit the data output by adding a command line argument as a single '.' separated string with the following entries:

[Processing Engine].[Region].[Site Code].[Solution Type]

All 4 elements must be in place. A '*' will act as a wild card.

For example, to get all RTNet processed data for site code 'WHC1', the following command would be used:
node rt_client.js RTNETJSON.*.WHC1.*
To access the Trimble GSOF stream from site CACT, use the following:
node rt_client.js TrimbleJSON.*.CACT.*

On linux and Mac OSes, an easy way to get a feel for which sites' geoJSON streams are currently coming through is to let the following command run, say for 20 seconds:
node rt_client.js *.*.*.* | grep topic > current_topics.txt
and then:
sort current_topics.txt| uniq > current_uniq_topics.txt

AttachmentSize
rt_client.js.txt1.83 KB