Transit starts crowd-sourcing real-time transit times as official feed breaks | Hacker News Reader

Transit starts crowd-sourcing real-time transit times as official feed breaks

medium.com

219 pointsant6n72 comments

NextBus is an old system that uses 2G cellular links. It doesn't need much data, so that's fine. Now 2G service is being discontinued by AT&T in SF, and Muni has to switch. Or switch to T-Mobile, which is offering 2G service for another year.

This was an early "Internet of Things" project. I met the inventor once at the Hacker Conference. This is a general problem with "Internet of Things" products - you're dependent on infrastructure which might go away.

There's no excuse for this happening -- AT&T announced the end of life of 2G four years ago in 2012. Now Muni claims that the shutdown took them by surprise and they are scrambling to replace all of the old 2G equipment

https://www.sfmta.com/about-sfmta/blog/why-muni-arrival-time...

Simply put, the deactivation work that affects our vehicles started sooner than expected and outpaced our ongoing upgrade of all Muni vehicles to a new communications and monitoring system.

http://usatoday30.usatoday.com/tech/news/story/2012-08-03/at...

AT&T Friday [announced] that the shutdown of its second-generation, or "2G," wireless network will be complete by the end of 2016

At the time, I was managing some devices that used 2G to send telemetry -- our AT&T sales rep called us to warn us about the shutdown.

I leased an Infinity over a year ago... it has the same "problem": the connected features rely on an ATT 2G radio. They made me sign a form acknowledging it would stop working January 1, 2017.

There is zero chance Muni wasn't aware of this. I have no idea why they didn't ask for money to upgrade the system earlier... it seems like a tailor-made situation to get a budget increase: legitimately blame it on someone else and use the largesse to buy a brand new much better system that you normally can't get approved.

Sure there is.

> There's no excuse for this happening -- AT&T announced the end of life of 2G four years ago in 2012. Now Muni claims that the shutdown took them by surprise and they are scrambling to replace all of the old 2G equipment

Typically, what that means is, pulled out of the ciphered jargon, "Relevant technical information was not adequately conveyed to the appropriate management in such a fashion as to outlast political appointment churn, thus, the current execs had no idea".

It's a two sided problem: well meaning and ignorant top levels and wellmeaning and poor communicating lower management.

The other analysis is that the current leadership was hung out on a limb to dry by the people who did know it was a problem, and the current leadership couldn't surface the relevant information. :-)

Muni has been upgrading the radios in the shelter signs over the last year, so they're definitely aware of it and have been working on it.

They could use Hologram's SIM which uses T-Mobile's 2G and 3G network as well as AT&T in the US [1]

[1] https://hologram.io/cellular/

Given SF is more and more the center of the US, how is it that your public transport still falls victim to such neglect? I understand the lack of appreciation for most cities in the US (like mine) since we don't really prioritize, but SF should have its shit together given how dense of a city it has become.

Politics.

Given SF is more and more the center of the US

It isn't. And Muni has been awful forever, admirably steadfast in awfulness through multiple booms and busts.

Hahahaha, once you live here you realize the incompetence is pervasive. California and SF is a very mismanaged place.

Public transport sucks everywhere in America, and I have never heard SF described as the center of the US. What do you mean by that?

SF is not the center of the US. It's just a city in California.

There are two serious global cities in the US — NYC and SF. I don't think it is unreasonable to suggest that the center of power/influence/whatever in the US is shifting to these cities over middle America.

> There are two serious global cities in the US — NYC and SF

After New York each of Los Angeles, Chicago, Dallas, Houston and D.C.'s metropolitan areas are more economically larger than San Francisco's [1]. SF is also relatively negligible as a global financial centre.

[1] https://en.m.wikipedia.org/wiki/List_of_U.S._metropolitan_ar...

MSAs aren't great definitions of geographical areas — San Jose is part of the Bay Area. If you include San Jose in SF, then SF is the clear #3 in total — and per capita would be a step function ahead of any other American city apart from NY.

Even beyond this, I don't think GDP is a good definition of "global city" — I don't think a city with massive commodity outflows could fit into the category, for instance.

If you include San Jose in SF, you're not really talking about SF. I've lived in SF on and off for many years and am very fond of the place. The notion it's a 'global city' in the same category as NYC or LA seems like quite a stretch.

Um.

Los Angeles. Chicago. Atlanta.

There are many global cities in the US - NYC, Chicago, D.C, Dallas, Houston and S.F would maybe be a distant 6th or 7th. You might want go do some traveling and broaden your perspective.

How can you say that the center of power and influence is shifting to New York and S.F? New York has been a center of power since the Dutch arrived.

Besides a bunch of tech startups what is S.F the center of?

San Francisco's Muni is not subject to neglect! It is meeting its objectives very well. It's just that its objectives don't actually include things like "actually moving people" and "cost control" (or even "revenue"). They are more like "advance the political career of whoever is in charge", at the highest levels, and "provide lifetime employment for everyone in the union with incredibly generous terms of work", for day to day operations. One of Muni's main problems with timeliness is drivers who don't show up, requiring either overtime work from other drivers (which they have trouble affording) or cancelling the bus.

And if you think that's great, then just wait while the CALPERS pension fund (all state employees) collapses over the next few decades and no one has any money for transit anymore. It's not just the usual overpromise and underdelivering with a taxpayer guarantee on the difference. It's not even just a problem with corruption, explicit bribery, and pay-to-play campaign donations, though it has been wreaked with all of those scandals. Even when it's otherwise ostensibly on the level, it doesn't try to earn money for its employees' retirement so much as it tries, as the state treasurer once proudly put it, to "mobilize the power of the capital markets for public purpose". It'll likely make the bankrupt New York City of the 1970s look like a beacon of prosperity and fiscal responsibility.

(There's other incompetence at work, of course.)

That is an unwarranted assumption offhandedly disguised as a fact: San Francisco is not, nor will likely become, ”the center of the US” — and is even outranked by a multitude of supposedly second-rate urban areas in terms of economic relevance.

>"Given SF is more and more the center of the US ..."

How is a small city(124 km2) with a population less than 900K people, that sits on the extreme edge of the U.S the center exactly?

Geography and population not withstanding, it's not the center for fashion, the arts, publishing, politics, finance or even tech as that distinction belongs to the peninsula.

I don't believe the perception of S.F as the center of the U.S is even remotely common.

Muni doesn't run this directly; it's contracted out to NextBus, a division of Cubic Information Systems since 2013. The NextBus web site says their vice president of engineering is Molsen Fahim.[1] But his LinkedIn profile says he left that job in March 2016 and moved to Cubic in another role.[2]

NextBus's interim general manager is Blair Brown, and his LinkedIn profile says he's been "interim" since 2015.[3] He was previously chief operating officer of Temple Beth El, a synagogue, and previously held some sales jobs, including being head of sales at NextBus in the past.

So there's the problem. Cubic bought a working system, the technical people left, and the sales types now in charge can't keep it going.

[1] http://nextbus.cubic.com/About/Management-Team [2] https://www.linkedin.com/in/mohsen-fahim-7035a34 [3] https://www.linkedin.com/in/brownblair

There is currently work going on to revamp nextbus in Cubic.

Source: I recently left a cubic subsidiary that was involved in this work.

Is it really Cubic's problem? The problem seems to be on Muni's side with outdated mobile equipment on the bus. Where operators are not quite as stupid, such as directly across the water in Oakland, NextBus continues to work.

More info. Muni is getting a new farebox system which includes location reporting. The old system was from Cubic, which no longer makes fare boxes. So the existing system is abandonware and the new system, from Genfare (formerly Johnson Farebox, from 1880) isn't in yet. Genfare is part of the conglomerate SPX, which purchased Genfare in 2012 and for which Genfare is a minor product line. So they'll probably screw up that business.

[1] https://sfbay.ca/2016/06/28/new-muni-fareboxes-approved-by-s...

>Today we’re excited and proud to be launching in San Francisco. Not that we planned it or anything. Literally we just decided an hour ago to do this because somebody asked us to go live and we said “cool okay.”

Startup culture at its finest right here.

That statement actually makes me trust them less... I'm not saying they shouldn't move quickly on such opportunity, but don't make it so obvious.

I hope it works flawlessly.

Only reason we were able to deploy this quickly is because of the 2 pilots we've been successfully running for the past 3 weeks.

I don't for a second doubt the ability. I'm a big believer in moving fast, especially when there's such a bit opportunity for engagement and PR.

It's just the wording which I found a bit "loose". But I'm a stickler for that sort of stuff ;)

I love the competition between Transit and CityMapper. In cities with coverage, they're both so good it's stressful trying to choose!

We are spoiled by both in NYC and surrounding counties, in Paris Transit is crushing it with Velib integration, while in Copenhagen CityMapper's local know how is amazing.

Is this the same company? They list only supporting 2 Canadian cities (Victoria, Montreal) in the top link.

Those 2 cities is where we've been piloting crowdsourcing. The app itself works in 125 metro areas.

Waterloo region (including Cambridge and Kitchener in Ontario) is supported, as is Guelph. These aren't big cities by any stretch.

I see a part list here: https://transitapp.com/region and more cities listed here: https://play.google.com/store/apps/details?id=com.thetransit...

One of my lines I can ride into work is the N-Judah line. This line is the busiest MUNI line. It is faster to get to my work because it goes underground and the MUNI buses I have to ride go slower, especially when there are a lot of pickups.

It is the most frustrating experience to walk to the N line because it is such a huge gamble. It use to get so crowded that you had to shove in there like you were on a line going into Tokyo. Just wall-to-wall people. They alleviated some of that by having more stops come back earlier going the other way. Only took them 2+ years since I've been here to decide that was an actual GOOD IDEA.

Equally frustrating riding on the line is that the N-Judah is sometimes actually a J-Church train that comes back right before you enter the tunnel into Market St. The only sign it is not the one you want is the last car on the train is locked out, and the badging for the train still shows N-Judah.

So sometimes you get on that expecting to go through the tunnel except it dumps you right outside of it. Now you have to pray there is room leftover on the next N-Judah line. Or alternatively you can try for the J-Church train that is entering near where the N dropped you off. So N-Judah is the closest line, but 50% of the time I have to use the J-Church train to actually get into work.

And that's even assuming the tunnel into Market St. is not bogged down. Whose genius idea was it to have all lines converge into one line?

What's the point of all this and what's it have to do with this article? Well fucking MUNI that's what. They recently increased fees for cash payment/limited payment card to $2.50. It is still just "$2.25" if you have a Clipper card. When I arrived here 4 years ago it was $1? $1.25? to ride. Literally for the increase of $1 more per ride I have seen service get progressively worse. When this bug hit, it said 30 minutes 'til the next N-Judah train. I tweeted to MUNI's twitter about what was going on and it took them 20 minutes to respond. And all they can say is "technical" problem as though 40% of the population in SF doesn't know what that really means. It is an embarrassment. SF is suppose to be some tech capital of the world yet we can't even ride into work without pulling our hair out over getting there.

Sorry, ranting, but it is unbelievable how incompetent our public transportation is given how much money they have.

Do they really have that much money? Public transit has been chronically underfunded in most of the US, including progressive metro areas. Boston has billions of dollars of backlogged maintenance, which contributes a lot to reliability problems.

All in, about ~1.8bn annual budget in SF.

The Proposed Operating Budget for FY 2017 totals to $1,154.7 million[0]

The Proposed MTA FY 2015 and FY 2016 Capital Budget includes an appropriation of $562.9 million in FY 2015 and $669.0 million in FY 2016. [1]

Is that underfunded? SFMTA has massive basic accountability and prioritization problems, tons of more money wouldn't make those better.

[0] - https://www.sfmta.com/sites/default/files/agendaitems/2016/4...

[1] - https://www.sfmta.com/sites/default/files/agendaitems/4-15-1...

according to wikipedia, the SFMTA (the agency that runs MUNI) has a 35% farebox recovery ratio [0] (fares only cover 35% the cost of running the system).

[0] https://en.wikipedia.org/wiki/Farebox_recovery_ratio

One problem is that you can walk on the buses without proving you paid for fare, and the bus drivers have the right to ask but in practice never do. It would likely go up by a bit if they were more strict about it.

The 38-Geary is actually the busiest, but since it's broken up in the the local, "rapid", and express variants it doesn't look that way in the tables.

ETA: also it's been 14 years since the fare was $1.

I think you're misremembering what fares used to be. Muni raised single trips to $2 in 2009 (and now it's $2.50 in 2017). Meanwhile my rent has more than doubled in that time.

It's actually (still) much cheaper than other cities I have lived in, and anecdotally fare evasion seems extremely high. Relative to the cost of living here, fares should be like $5-6. Maybe with that kind of money they could actually accomplish anything, although I kind of doubt the problem is solely financial.

You are right. I was sure that I paid less than $2 when I first moved into the city but I looked it up and apparently it was $2 for a while before the hike.

The Transit app should pay Muni drivers to use the app. Maybe $5 to $10 a shift would be enough motivation. That would be an extra $100 to $200 a month income

I'd be surprised if they were allowed to use their phones while working.

I think OP meant so that they have data for all transportation vs relying on users crowdsourced. Still crowdsourced but with better paid sourcing :) Note that it's passive - users don't have to do anything I believe aside from having the app.

Yes, this is what I mean. Drivers would not have to do anything if Transit is smart about it. "Looks like you are a driver. You have taken two return trips on the 14 today. Enter your debit card and keep the app installed and we'll pay you $10 every day you drive"

Homeless people could just sit on buses all day and get paid for it too. And within a few days some disruptive people will duct tape their phones plus large batteries to the bottom of buses. Then terror alert.

The homeless part sounds like a pretty good idea. You hear a lot about startup employees looking the other way with regard to social problems in the Bay Area, so partial solutions like this could be interesting? Of course $10 per day doesn't really raise anyone out of homelessness...

Good idea? It's political suicide! Their name would be dragged in the mud faster than you can say "exploitation". You can only offer low payments (that is, in exchange for something, not donations) to people who don't really need them, never to the homeless or otherwise unemployed.

Muni should be the one paying Transit. Transit actually makes Muni semi-tolerable -- at least you have an idea of how much it will suck. And I'm sure NextMuni is not cheap. As far as I know, Transit's only revenue stream right now is from Uber and maybe bike shares.

This is a smart use of crowdsourcing. From the app you can know the number of people who are waiting for the bus and relying on your reporting. Kind of gives you a sense of community. I hope this takes off.

Right. Their little smiley telling me that nine people are being helped by my activating their app kept me longer at the stop than otherwise. The bus was four minutes off their prediction, though.

You're only going to get accurate departure times if someone else is using GO on that bus/train you're waiting for!

Oh, yes, I know. I hope the fellows downstream had better performance.

What does it take to add real-time data for a new city?

I've been interested in trying to scrape the data from the transit provider here. They have what appears to be reasonably accurate arrival predictions which they display through an atrocious app, and electronic signboards at some bus stops. The busses have a terminal for the driver that displays how long they need to dwell at times stops and announces the next stop, so I assume this is (at an unknown interval) transmitting the locations to their server.

I haven't gone further in looking at it yet, because google maps will only add data sources directly from the official transit provider.

I live in a major city of ~800K with one of the lowest public transit ridership in the US, and I've learned to not quite trust their time predictions. We do luckily have GPS location data, and I regularly just look at the bus' actual position to gauge time most of the time.

Neat. I had a similar idea a few years ago to make NextBus times more accurate... but fell down a rabbit hole trying to implement it as a cross-platform HTML app and never actually launched.

If anyone wants my idea: My perception of NextBus's arrival times are that they're based on a very naive algorithm. I think it might literally be assuming that all buses always move at an average 15 mph. Lots of ways to make that more accurate with collecting just a teensy bit of actual arrival time data.

I don't know about other systems but I've spent time in the field analyzing NextMuni's predictions and they have no predictive power whatsoever. The first arrival time is weakly correlated with actual arrival, and the 2nd and 3rd times are not even correlated.

It's useless, essentially.

I learned early on to discard the units when using NextMuni. The J will arrive in three. Three what? Nobody knows, but there will be three of them.

Back when NextMuni was only 99% worthless, Transit had a method of actually getting reasonable arrival times out of it. Since each bus has a published GPS location (in addition to the NextMuni time), you can use that with driving direction data to loosely get travel time.

When the app did this you would initially see Next Muni times -- 2 minutes away! -- which would update a few seconds later for lines you were particularly looking at -- oh wait 20 minutes away!

Apparently NextMuni either just displays the schedule (which is literally never right) or does some kind of extremely basic estimation like you said. Many times I have seen a bus arrival time listed that was physically impossible -- a bus a mile away arriving in 1 minute, for instance.

Transit's method still fell down in the case of buses which appear on the route but are just sitting at the beginning of the line. It treats that as a bus that's en route, but in my experience they just hang out for as long as they feel like it. I learned to look up the locations on the map and discard those, just annoyance #475 of public transportation.

I tried downloading the app, but the app store was offline. Not sure if that's considered ironic.

With all of NYC's subway stations receiving cell reception as of less than a week ago, this would be amazing for train times here.

Besides a couple lines, the train times here are all estimated—and essentially useless.

They're working on a beacon-based train time system now [1]. Bluetooth beacons are installed on cars with trackside sensors as trains enter and exit stations. Station wi-fi is used to relay data about entering and exiting trains. Since the MTA already has train speed and track length data, it gives a pretty accurate guess at where the train is between stations.

It's not as accurate as full trackside sensors throughout the tunnels, but much faster and cheaper to spin up. Pretty clever overlay IMO.

[1] http://www.metro-magazine.com/rail/article/715387/nyc-transi...

This looks nice, but why does a bus app need to read my calendar? And photos? And identity?

Calendar to show a "Show Upcoming Events" option in the search. If you don't tap it we don't access it. Photos to allow installing the app on an SD card (popular request). Identity to use your name in the Send Feedback option. Here again if you don't tap it we don't access it.

In the latest version of Android you can turn these off under Settings->Apps->Transit->App Permissions

Very impressive. The word 'agile' has bad connotations, but I think it applies here. I hope that Transit and others can use this to jump-start their service in smaller municipalities. My hometown's bus system isn't terrible, but they're not the kind of organization that would see the value in making a proper app, or by publishing real-time bus data. I would love to be able to use an app like that where I live.

Nifty, I know moovit has been doing this for a few years now as well as crowd sourcing route information for a bit. Transit has a pretty nice UI and a few interesting integrations. Fun to see some competition in this app space.

Twice in the last few days I tried to use NextMuni and it gave me super wrong estimates, way more wrong than I've ever seen it. I wonder if this was why. I'm really grateful to have another option available.

I like this, but then it highlights the underground train stops without mobile signal.

Checking, I see a lot of "invalid dates" alerts on lines.

This should be crowdsourced. The official data here in the "T" bus service area is unreliable. Crowdsourcing would also enable measuring real-world service performance.

SF public transit continues to boggle the mind.

Reply on news.ycombinator.com