Recently, ESRI published their new mobile augmented reality ArcGIS runtime for Android and iOS. As the main railway company in Switzerland (#sbbcffffs), we do have digital twins of many of our assets. GIS is one of the technologies we use in many areas to work with them. So we got excited when ESRI told us (well, they told everyone on the Internet), that we could also visualize those assets in augmented reality from now on.
In the first part of this story I’d like to show you why this can be a great deal for us as a railway company (aka the Business Case) before I’m going to share our insights of working with this new framework.
GIS in a railway company (or why no train would be operating without it)
GIS stands for Geographical Information System and basically describes anything that deals with data containing spatial references. In our everyday life GIS is omnipresent, one of the most prominent examples being Google Maps. So, why might this be important for a railway company? They hopefully never need the routing features?
In fact, without GIS not a single train would be operating in Switzerland. And for the sake of this article, I’d like to give you the following example: We are using GIS for the maintenance of all the green spaces along our railway tracks. Our forest rangers permanently monitor the condition of all trees, bushes and other vegetation inside a 100 meter wide corridor along the railway tracks. If a tree is posing a risk of falling onto the track, the foresters commission external companies to take care of the tree. Obviously, this is a never-ending task. To help them not loosing track, we support them with a GIS-based iPad application, providing monitoring capabilities and information about the current condition of all green areas.
Now, if a forest ranger decides that some grassland needs to be cut (this usually happens once or twice a year), external foresters will come at night with some rather heavy duty machines for cutting back the grass.
Now, if you look closely at the GIF above, you’ll notice that the cutting arm is moving around to avoid obstacles. Unfortunately it’s very hard to see all obstacles at night and machine operators “cut” into them unintentionally. This is definitely bad for the blades of the cutting arm — those blades don’t come cheap. So we decided to warn them using our iPad application once they come near an obstacle.
Unfortunately this wasn’t enough and machine operators still hit obstacles from time to time. One of the issues seemed to be that the real-world map was of little use at night, since all those visual reference points were still hard to see. However, there seemed to be a huge potential in showing them obstacles right in their field of vision.
Our experiences with ArcGis Augmented Reality
Enter the new augmented reality functionality of the ArcGis Runtime SDK released October last year.
Since we already had all the necessary data on hand (on GIS feature services) we decided to give this new framework a go and so three of our iOS developers invested three days to explore the potential (and eventual limitations) of it. We started out by visualizing railway tracks in augmented reality but soon noticed, that buildings were way easier to refer to visually. The setup is fairly easy if you have some experience with the ArcGIS framework. Within 3 days we accomplished the following:
- Displaying of data (from our online feature services) in augmented reality
- (Un)hiding of different features
- Tapping on objects in augmented reality and showing corresponding information
Now, this all sounds great but if you have watched the video above you will certainly notice, that there are still some (major) issues. To make augmented reality work nicely (meaning that all objects are displayed where they should be) there are two prerequisites:
- The app needs to know where you are (usually this is done using the GPS)
- The app needs to know in which direction you are looking (this is called heading and it’s usually done by sensor-fusion of compass, gyro and acceleration sensors)
If one of those informations is not accurate (and I mean really accurate), the following issues will emerge.
Issue 1: GPS position not accurate
Generally speaking, GPS positioning on mobile devices is not very accurate (you get precision within a few meters at best).
- Even an imprecision of 1m is problematic for objects which are close to you (they will appear closer or further away from you)
- For objects which are further away from you this is less of an issue
- Indoor-localisation is not possible with GPS, obviously
Issue 2: Heading not accurate
The same applies for the estimated heading of mobile devices. This is really hard to achieve (especially avoiding drifting over time).
- Objects appear vertically or horizontally displaced
- This can be manually corrected (we implemented this by dragging sideways). However the heading will still drift over time.
Conclusion (for now)
There is a lot of potential in displaying GIS data in augmented reality. However, at the time being we won’t be using the ESRI ArcGIS provided functionality due to the GPS/heading inaccuracy. Sure, this might improve a little over time, but for now it’s not accurate enough for our use case.
On the other hand, we made very good experiences with augmented reality, where your position is calculated using machine learning and more specifically image recognition (check out our SBB AR App). Unfortunately, this means that you a need a 3D image model of your surroundings which is reasonable for closed spaces (like train stations) but not yet for open-world scenarios.
