No GPS required: our app can now locate underground trains
blog.transitapp.com
blog.transitapp.com
Edit: found it. https://medium.com/snips-ai/underground-location-tracking-3e...
Edit: at least the excellent Physics Toolbox Sensor Suite gives me a barometer signal indicating 102570 Pa right now.
When you remove the filter, it returns 12177 results. So only ~5% of phone models include a barometer.
146/1109
now correct it for number of sold phones each (estimated by looking at the 70 most popular models) and we'll get why they said 1/4.
That seems unlikely in an environment with no cellular / wifi signal. Theoretically possible, but expensive for battery and probably disallowed by the OS.
I'd have to have missed the title of the post, not read the post itself, not read GP's comment, not thought about why there'd be a pressure change, to have missed that particular detail. I appreciate you trying to be helpful though. :)
I wonder if it'll Just Work [eventually, given enough repetitions], or if the crowd-sourced network location algorithms will filter it because it is dynamic.
A trivial example would be people on different floors of a skyscraper—although I suppose gps works poorly indoors anyway. Still, even outdoors there are peaks and crevices, and on a steep slope a very trivial change in lat/lon could lead to a major change in altitude.
https://www.ncesc.com/geographic-pedia/why-is-my-gps-elevati...
But elevation maps are not detailed enough and position is not accurate enough to get accurate elevation. Think about standing on trail along steep slope. The position not being that accurate is fine since you know you are on trail. But altitude could vary wildly going up or down slope, or even up or down trail. It is probably similar to GPS vertical accuracy, but were going for more accuracy that barometer provides.
Because the SATs only give you a pseudorange distance between you and the sat, so each say is most accurate solving for distance to/from that sat, and much less helpful resolving angle to the sat.
With a clear SkyView, around just under half of the sats are hidden by the earth.
This means that you get a full 360 degrees of data that can be near the horizon helping resolve lat/long.
But only about half of that sky is helpful for altitude. The birds you can't hear directly below the earth would be the most helpful for improving the altitude fix if you could hear them.
Baro is handy because you can take the absolute altitude from GPS as a low frequency baseline and use the baro for high frequency changes. Then when GPS says we teleported +200 feet when a new sat comes into view, we can temper that that with baro information.
You also probably want an accelerometer.
Note that's "sensitive", not "precise". The atmosphere changes pressure with the weather, so you don't get accurate absolute altitude - at least not without knowing the local "pressure altitude". Barometric pressure can vary in a way that represents plus or minus 300 or 400m of altitude, and can swing between high and low fairly rapidly in extreme weather events.
But over short timeframe the change in barometric altitude can be very useful. Glider pilots (including paragliders and even RC gliders) often use very sensitive barometric pressure sensors to detect when they're in rising to sinking air, down at the few meters per minute range of sensitivity.
[1] https://www.nstb.tc.faa.gov/reports/2020_Q4_SPS_PAN_v2.0.pdf...
The problem is, it was basically useless. The main use case for heart rate monitoring is continuously throughout the day/night, or during exercise. A watch is very good at this. An optical sensor on the back of your phone is not.
Periodically checking your heart rate by holding your phone in a specific way is not a useful feature for that many people.
Turns out, when you have a known luminance, white balance, and frame-rate... the DSP to grab heart rate from a finger is trivial.
But on the Galaxy S7 that I think is being alluded to here, it was definitely a separate sensor -- a MAX86902, IIRC.
Conclusion, Fitbit and Google heart rate monitors on those wearables are hot garbage. Cue some snooty googler insisting I'm doing it wrong somehow.
As others have stated, not all phones have pressure sensors, and the quality of the readings also varies a lot between different models. For example, we had one device where the readings would spike when squeezing the phone.
Transit also doesn't have permission to read the pressure sensor, and our use case wouldn't justify asking for it.
For others wanting to implement similar functionality, it would be great if they didn't need to re-do the entire work from scratch.
I am actually currently working on a project to record the sound of the London Underground passing under me.
We can very clearly hear the Northern Line under us. It's < 30 meters below us.
I have become obsessed with getting high-quality, low frequency recordings of it passing under us.
Why? I don't know. I just can't take my mind off it.
For example, there are two tunnels (north and south bound). By correlating it with actual TfL data, can I figure out the sound signature of each?
More intriguingly, I know that there are maintenance vehicles that operate under us in off hours. Can I "catch" them?
I'm not sure what else I might do with this project, but the idea of capturing the sound of this semi-ephemeral creature that operates below me has captivated me.
I'm interested in extremely weak, high-frequency vibrations of everyday things while in resting state (which is sort of the opposite of what you're after, as I understand), but have not gotten far in acquiring the sensors. I'd love to get a laser doppler vibrometer, but they're pricey.
I know very little about audio engineering. I wonder what else I might be able to use to pick up the vibration signature?
I completely understand your drive to pick up those high-frequency vibrations! There's a whole secret world of vibrations out there, and we can analyze it!
Could you use the accelerometer on a smart phone to 'hear' the vibrations?
However, what you're saying is completely legit. I read that I might want to lay a large, sturdy thing on the floor and put the mic on that, for the same reason that you gave. I used a large old pane of glass that I found in the cellar.
What do seismographs do? People also use acoustic pings to measure soil and rock density underground. What are those devices using for their “microphone”?
And yeah, would an accelerometer work? I don’t know what the temporal resolution is on your standard Amazon / AliExpress accelerometer is but it’s probably pretty decent?
Amplifying and digitizing the signal is straightforward for an experienced EE, but you might look into acquiring some Raspberry Shake hardware which has it all already done.
Maybe something like that could be turned into a reasonable microphone?
I wonder what other ways there might be to detect it.
Someone else mentioned literal vibration, but I figure a contact mic would already pick that up – I mean, sound is already vibration...
https://www.ltmuseum.co.uk/collections/collections-online/in...
Whoever wrote this did a fantastic job.
(Although in fairness they so often get this laughably wrong, and cycle through useless "remember your luggage" style messages so you have to wait like 20s to see the information you want - critically bad on a train.)
I will literally volunteer my time to canvass for anyone who runs on busting NYC public sector unions
Now accepting funding offers at $500m post-money valuation cap.
The magic formula: "if is_train_moving, countdown to next station"
- human driver
- train capacity and current load
- train model (or any powertrain variations)
More reliable might be to check rail track features from accelerometer: tilt, turns, bumps, or a combinations of everything. Even sounds on turns, changes of backround during merging tunnels, etc. Integrated acceleration gives train speed, which is also useful along with other inputs.
I think I’d struggle with finding a signal through all that noise.
Accuracy is poor when blind, but if you have any info about where the user last had a location fix it became very good.
Neither company found sufficient interest to deploy the sensor.
Not in any even remotely modern system that will have a high degree of automation and if there's a driver, they're only closing and opening doors, and telling the train to get going.
I assumed that everything would be automated but in fact nothing is.
They explained that they regularly face situations that require human intervention (several times a day), and that even a little automation would reduce the level of attention on the part of the conductor.
The conductor is required to interact with the controls at least every 30 seconds to avoid setting off alarms.
So I think it is a much, much harder environment to automate. Paris do have some metro line that are fully automatic (line 1 at least) and both Rennes metro lines are fully automatic. It is much easier to control the environment around a metro and ensure that nothing can go on the rails, and have surveillance system to check if, if it does happen, it is detected ASAP.
Or camels: https://www.francetvinfo.fr/animaux/un-dromadaire-apercu-aux...
> So I think it is a much, much harder environment to automate. Paris do have some metro line that are fully automatic (line 1 at least) and both Rennes metro lines are fully automatic. It is much easier to control the environment around a metro and ensure that nothing can go on the rails, and have surveillance system to check if, if it does happen, it is detected ASAP.
Even the non-fully automatic lines use heavy automation (e.g. lines such as 2, 3, 5, 7, 8, 9, 11, 12, 13).
https://www.networkrail.co.uk/running-the-railway/looking-af...
Barcelona, on the other hand, would qualify for everything except line 4.
Also if we want to quibble, at least a few sections of the Bakerloo and Piccadilly lines (which are still completely manually driven) were opened after 1906.
From 1907 onwards every newly built line was automated from opening (though indeed extensions or "new" lines made by splitting an existing line were sometimes manually driven).
(The Jubilee line in 1979 – which started out being manually driven – wouldn't count according to your criteria, because part of it was "made by splitting an existing line", and apart from being heavy rail, arguably Crossrail could be argued to be an extension, too, even though it does operate automatically on the new-built bits, i.e. the central section plus the Abbey Wood branch.)
Paris has the same, and also a number of lines (1, 4, 14, and soon 15) which have no driver at all.
NYC Subway is an outlier in how obsolete everything is.
Grand Paris Express' president also talked about this, and compared the 2nd avenue subway to the Grand Paris Express (well, one cost $4.45 billion for 1.8 mi / 2.9 km and 4 stations; the first line of GPE, 15 south is 33km, 16 stations, and costs ~8 billion euros so it's a really bad comparison), and has said that if he had to do things the way the MTA do it, GPE would have gone nowhere.
The real advantage of automation though is without a human driver it becomes cheap to run a lot of trains that are not very full and that makes your system nearly as convenient as driving in the suburbs.
All modern ones (in an urban/suburban context, which is what the post and thread are about; interurban is a bit more mixed, but every high speed line is highly automated too, because at those speeds there simply isn't the time for a human to react) are.
> The real advantage of automation though is without a human driver it becomes cheap to run a lot of trains that are not very full
And you can also have a very high frequency (60-90s intervals), which is impossible with manual operations. This increases capacity, on top of the convenience.
They have a very distinct hum that matches the rpm of the wheels. I once built a crude speedometer using SFTs, peak detection and kalman filtering.
I figured someone was working on this but it’s so refreshing to read about the thought and level of detail that went into your design. What an effort too! Congratulations transit team, you all should be so proud for solving what I’d imagine is one of transit’s largest small gripes.
I do love this "Transit" app though.
Or is the hardware in smartphones too inaccurate even with the extra information?
The thing that should help the most would probably be the hall sensor/magnetometer/compass. That should output decent dead reckoning.
Doing this just with the gyroscope will work very well for short movements, but it will be close to useless on long, gentle curves. Unfortunately, those MEMS gyroscopes drift quite a lot over tens of seconds. Not a problem if you can do sensor fusion with the magnetometer (reckoning) and the accelerometer (where is "down"), but the latter can't be used on a fast train, acceleration/deceleration of the vehicle and forces in curves make finding gravity challenging. No idea how well a compass works inside a subway tunnel.
But maybe I'm wrong, I just have experience trying an "artificial horizon" app on an aircraft - and here, the accelerometer is completely useless for "down". A single maneuver with some Gs and the horizon has no idea what the pitch angle is. Noisy magnetic environment, GPS off? It also doesn't know where it's going.
It says they use the accelerometer for detecting the user's current mode of transportation, not the distance.
It says they use the train schedules to figure out the current location of the train, based on when it started moving and how long it's been moving.
They don't mention anything like dead reckoning anywhere, unless I missed it?
So focus on speed not acceleration.
Sort of related, what I find kind of silly, is that my car probably knows really well where I am, but can't help my phone figure out where my car is pointing to despite being connected to it.
My point is, I see only one real business case for this.
[1] https://play.google.com/store/apps/details?id=com.thetransit...
If you're just putting in a destination and getting a fastest route (much like you'd do in Google Maps), then I didn't see much benefit to the premium version.
The joke was _two_ sentences, and I, for one, appreciated it.
If you're going to quote regulations, make sure to bring a gun to a gun fight!
- it has no useful meaning in a media you can resize.
- adding dots (.) does not noticeably change the meaning, you could replace with a comma : "Oh thanks goodness, I new…", allowing cheating on the fight rules. I don’t think that’s desirable way we want to interact as a HN-user-community (personal opinion)
- taking the literal read anyway, I see one line and two sentences right now. It’s a one liner. (not English native, may I missed a secondary meaning?)
Dang clarify it bellow the post in linked and cite scott_s, none of which talking about the length of the joke but both referring to noise.
I personally also found it funny BUT I also saw like 10 substanceless not-so-funny jokes today on HN. INHM problem is not the joke itself but the emptiness of the post if you take the joke aside.
Which is good, IMO, since one important property of a social system is flexibility.
Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
Some could feel my comment escape them from it’s tone and it will be wonderful if you or someone else share a better way to say what I said. I’m doing my best to not hurt others, which sometimes is seen as coldness. Which I admit is not kind.
Vote accordingly and proceed onward. If collectively a negligible fraction also do the same then the results might just surprise. Ostracism is a powerful tool, while not always used for good, it does serve a purpose in society.
Here's to the future!
Google is a huge advantage here, they have both the spatio-temporal intent of the user as well as the physical flow feed. There is so much position data flowing off android phones, that they are able to see the whole topology.
I see they support the the largest cities in Sweden and Norway, wonder if there are any plans for Copenhagen, Denmark?
Especially if you know someone at the transit agency or can help them even in a very small way.
Our conclusion was that the feature didn't work in the danish metros for reasons we never got to deep dive into. It's most likely related to the fact that many of the metro stations are built in concrete, as such there's no GPS data in most of them unless you're very close to or on the surface and no motion data.
I'd be surprised if they got this particular feature working but who knows... maybe if we had looked into the raw sensor output we might have been able to work something out.
In the end we made a solution to help determine when you're moving or not by utilizing beacons.
How come there's no motion data?
1. Seems a bit weird to be looking at the accelerometer data yet miss the obvious approach of summing up the acceleration to get the velocity and then summing that up to get position. Yes I know about drift but even then I'd assume the fairly constant several-second g force of pulling in or out of a station or taking a curve would be a strong signal easy to distinguish from short lived jostling.
2. The "train moving" frequency you discovered via fourier analysis was most likely the hunting oscillation. This has to do with how the wheels of the train are designed to force it to turn opposite to any deviation from the direction of the track. Thus there is a back and forth "hunting" for the center that is completely determined by the geometry of the track and the wheels, and therefore the length of track per complete back-and-forth cycle (aka the wavelength) is constant. The frequency of the oscillation (aka back-and-forth cycles per second) is just this constant length divided by the velocity of the train. This fact could be leveraged to estimate the actual train speed rather than just moving/not moving.
3. Combining 1 and 2, a combination of integrating acceleration and confirming / correcting estimated velocity with the expected hunting oscillation would likely be the most powerful / reliable model.
4. Using a classifier seems overkill here. But I'm sure at some point it was easier to just raw-data it than work out a theory driven model which accounted for all the practical confounding factors.
You also have to assume that GPS has already been lost at this point, so you have to do it from the departure location of the train at the right time.
https://en.m.wikipedia.org/wiki/Inertial_navigation_system
Semi-related but just as fascinating: https://en.m.wikipedia.org/wiki/Terrain-following_radar
You are correct in saying the low frequency acceleration from starts, stops and turns can be distinguished from the higher frequency noise.
One big challenge was with orientation. Acceleration can look the same as deceleration and turns from the sensor's perspective, if you turn the phone around. Taking the integral of the gyro reading, the error would grow quadratically, and we found magnetometer readings unreliable depending on the vehicles.
Your point about the hunting oscillation is interesting and I agree, estimating the speed would be a great improvement.
Yesterday night and this morning it kept telling me to get off the train either two early or too late, and this evening it didn’t even think I got on the L in the first place. The app even lightly scolded me for “missing” my train!
The motion detection might be a convolutional network or an svm. The mixer model perhaps a classic neural network.
I don't believe any of those are encrypted and transmit ~1000-10000 time per second on 1500-1600mhz spectrum which is fairly simple to reproduce using even a cheap SDR kit.
It could also work with 0 input from the train's telemetry - in much the same way as the app - the device would get a reference gps signal (or wifi/BLE when it knows it's in the station), then with a built-in accelerometer (which it has the luxury of direction + stability if it's mounted in the train car) it can determine with greater accuracy where the train is and how far it's moved.
Spoofing GPS as a commercial operation is a quick way to get your company crushed by regulatory agencies.
On a serious note, I recall that smartphone location in metro in my city started to "just work" as soon as all stations and tunnels had indoor cell towers. Suddenly all apps worked fine and I forgot that problem ever existed.
Modern location detection is as scary as it is amazing.
Sounds to me this could be very complicated and expensive. I wonder if it would even be possible because you'd need to have the same signal spoof the correct positions to everyone who hears that signal.
If the entire sphere where I could be fits within 5m I guess I don't need other satellites and their time to start intersecting spheres.
How could it be possible to determine the location from a single timestamp and information about which satellite it belongs to? I suppose if the recipient already has a fix, then it could perhaps survive with less than 3 satellites by making some assumptions, but I imagine this will result in lower quality location information.
Were you proposing to assume the location of a 5m sphere the recipient is in?
I'd assumed the GPS calendar was somehow broadcasted by the GPS network too, which kind of means that they also share their location.
> Were you proposing to assume the location of a 5m sphere the recipient is in?
I guess the proposal was to change the problem from pinpoint a single point in space, to figure out roughly where I am, in which case being anywhere in a tiny sphere is pretty much the same as being in a point after you account for errors.
GPS spoofing should not be done in my opinion until the negative side-effects are well understood.
https://www.leparisien.fr/info-paris-ile-de-france-oise/tran...
Knowing the next station is usually a solved problem that doesn't need a smartphone, because that's displayed in the train itself and called out on speakers. But once you are on the platform and you need to ask the route planner what the fastest route is to a specific station (It could be walking to the surface and taking a bus, so it's not as simple as looking at the subway map) - then you are out of luck if platforms don't have 4/5G coverage!
I suspect the UK just doesn’t have modern standards for mobile internet service, as it isn’t limited to TfL services in London.
There isn’t even reliable mobile coverage on stretches of the West Coast Main Line rail corridor in England, which is outdoors and connects the UK’s most populous cities.
Much of London isn’t particularly reliable either on most mobile networks, in my experience. Meanwhile, in cities like Prague or Washington I can often get mobile download speeds outdoors of 400 Mbps - 1-2 Gbps.
That probably influences the decisions on roll-out and cost.
TFL has been ahead-of-the-curve with the rollout of some tech (Oyster cards, tap'n'go), but not so much with others when they get hacked.
Here, they seem to estimate which stations they are at or between, and use that as absolute references.
Your running app is using accelerometer data to recognize when you're taking a step, and combining that with GPS data to measure how far you're running. So if it measures 100 meter using GPS and counts 100 steps during that time, your step length is 1 m. Garmin watches uses this to measure your running distance on treadmills, but they want you to calibrate it by running with GPS on a flat surface outdoor first. I don't know if any watches or apps use this in combination with GPS to make better position estimates when the GPS reception is poor, my guess is no because it's really only useful for distance, not position, and generally when running outdoors you're not running in completely straight lines. But it should be possible to estimate distance and combining it with their routing software to guess which way you took for short parts of the course.
Would be useful if I could teach this to your app.
I’m curious about the failure cases. Are they caused by exceptional circumstances, such as the train moving more slowly than normal or skipping a station? Or when you unexpectedly catch an express train or go the opposite direction? Does the algorithm know that it doesn’t know where you are, or does it confidently tell you the wrong station until the GPS is acquired?
I do love this "Transit" app though.
Other difficult cases include trains stopping between two stations (doesn't happen everywhere, but it's frequent in NYC), or a user walking fast onboard a moving train, which can be mistaken for the user having gotten off the train.
Taking a train in the opposite directions will break the assumptions we make and we won't know until the next GPS location
I have noticed that a year ago or so, Google Maps app would lose the GNSS signal and stop updating the position while there was no signal. But now I have noticed that the position is updated, although is not accurate. I wonder if something similar has been implemented...
(Of course, that one area of superior performance couldn't make up for the awful GUIs, the awful resistive touchscreens, the $100 map updates, and suchlike)
Wildly inaccurate even with GPS it seems.
Instant uninstall. Sorry.
Not doing so would be astonishing given how many people use public transport here, and that TFL’s data is really, really open + easy to use.
That being said, if this app could convince cities to also be used for payment that would be a game changer. Uber for public transit would really remove so much friction from using transit.
A bunch of the larger / better-funded systems are also moving to just accepting credit cards directly on the readers, which is even easier.
Just show the predicted location of the train they should be on separately from their last known GPS position. Of course it would be difficult to market that alone as a novel innovative AI feature.
At least in NYC it should listen for door beeps.
This probably is wrong if trains for both directions stop at the same time from opposite sides of the same platform and you get on the wrong train. However this is a rare enough case that we can ignore it - but very annoying if you are that person (this case happens to most transit riders once or twice in their life). Even when this happens there often is other location data scattered around so you would likely only ride a stop or two before the app realizes you are on the wrong train and can reroute you.
I'm guessing this app won't work that well there. In fact it would probably generate false positives when labeling stations... ?
ML level - for some reason, made me think of ITER / fusion research trying to predict plasma behavior with ML also. Any specific connections people in the know care to point out?
One correction though - there is no subway line that goes across the Queensboro.
I found this to be a really well-done video on using quantum physics to track location integrating upon acceleration
I was excited to try it out but bummed to see my city isn't listed. It's even more disappointing considering Ulm Germany (with around 100k people) is there, but Cologne Germany (with a population of about 1.1mil) isn't.
There are lots of potential users here, especially since the official apps are aweful.
I think the answer is still unclear since they are using pre-determined routes (easier to track east -> west or east -> southwest than it is east -> north -> south -> north -> east again). But this is very cool that they have so much of the work done already. Maybe even all of it? I don't have the code to look at so ¯\_(ツ)_/¯
Either way, still freaked out they read my mind lol
This is a pretty cool use case though since as you said it would be much easier over a fixed route.
Even without gps data, just having access to your phone accelerometer is enough to give a lot of data about your life. Cumulated with Google insanely big amount of data about wifi access point location, it means that they know where you are even without gps activated and how you got there.
https://en.wikipedia.org/wiki/Dead_reckoning
The first car navigator, the Etak, came out in 1985 and used dead reckoning and quantization to tell where the car was on the map; see this excellent article from 2017:
https://www.fastcompany.com/3047828/who-needs-gps-the-forgot...
Today dead reckoning is used in aerial navigation, and commercial planes (and others) are equipped with Inertial Navigation systems to supplement GPS information; they are getting more and more precise but can still go wrong and need frequent re-calibration.
The next step is "Quantum Positioning System" that promises to detect infinitely small movements and produce perfect dead reckoning at all times, with a precision of the order of one centimeter. It has already been tested successfully. For now the machines are heavy and extremely expensive, but it's imaginable that in some not-so-distant future the technology will be much more available.
https://newatlas.com/aircraft/quantum-navigation-infleqtion-...
[0] https://timharford.com/2023/07/cautionary-tales-the-v2-trilo...
But dead reckoning for train travel should be massively easier. Train movement is constrained to tracks so you only need to resolve how far along track train has moved + possible junctions, which should already make the problem much simpler than e.g. airplane ins where you are resolving full 3d position. To make things even more easier, you only need to reckon between individual stops which prevents error accumulation over the whole trip. Lastly you have schedule information, so you know roughly where the train should be at any given moment.