Actually, not generated by AI. It was largely handwritten.. but since you found this phrasing objectionable, I removed it.
62 karma · joined December 7, 2011
The metric we actually care about is closer to "cost per accepted outcome" which would roughly be "total inference + retries + failures + the human effort required to verify, correct, or resume the work."
The first few are easy to instrument.
But "human effort verification" and “accepted” are much harder to define consistently across different agent workflows.
For our code review product we have a decent proxy because we can measure user reactions and whether findings are actionable and if it's merged/accepted.
For more open-ended agent tasks, I don’t think the industry has a particularly good answer yet... I would love a point if there is one?
The cheaper model isn’t actually cheaper if a human has to spend the savings babysitting it...
It spawns agent CLIs (Claude Code, Codex, Cursor, GitHub Copilot) with and without Unblocked's MCP server attached, then statistically compares the results: https://github.com/unblocked/unblocked-harness-compare
We likewise measured token savings, (wall clock) time, # tool calls, and # turns.
We certainly don't think of buddybuild as a closed ecosystem.
In fact, our users are able to customize buddybuild infinitely to meet the needs of their workflow (details can be found here: https://www.buddybuild.com/blog/customizing-the-build-proces...)
Further, while buddybuild is favored by thousands of because it can can auto-provision iOS devices for you on the fly. Meaning, you never have to deal with provisioning profiles and UDIDs ever again.
We also care deeply about making sure that teams can involve and engage their end users as part of the development process. One of the reasons we built buddybuild is because of the frustration sending builds to our users... AND then never hearing back from them. Phones are great devices for "consuming content", but it became obvious that the physical form factor of the device tends to result in infrequent feedback which was often ambiguous.
So, we build an SDK that gives you high quality and actionable bug reports from beta users. With a simple screenshot, testers can file bug reports that include graphically annotated screenshots, feedback notes and their device metadata.
We also have a feature called "Instant Replay", which lets you watch a video of the exact steps required to reproduce a crash. Instant Replay works in conjunction with our Crash Reporting solution to offer you both the exact lines of code that caused a crash as well as the exact sequence of events that lead up to it (https://www.buddybuild.com/blog/introducing-instant-replay)
Thousands of teams with fairly complex needs use buddybuild every single day. They teams like Slack, Meetup, FireFox, Wordpress and others..
You can read why they made the choice to use buddybuild here: http://www.buddybuild.com/customers
We're a mobile focused continuous integration and deployment solution.
When deciding which platforms to support, we looked at current adoption for Native iOS and Android (this was no brainer), Hybrid (PhoneGap, Ionic, Cordova) and "emerging" platforms (React Native, Xamarin).
Ultimately, we decided to prioritize the others first. CI/CD service is applicable to any mobile development team.. but to your point, buddybuild is better suited to those other platforms, as those are the only ones we support.
In our experience (which also factors inbound requests), Xamarin is still relatively nascent as compared to the others.
That said, you can should expect to see Xamarin support soon :)
Also, fwiw, Microsoft purchased Xamarin only recently.
Of the teams looking for cross platform solutions (versus native iOS or Android), Xamarin comes up on occasion.
However, React Native tends to be FAR more common as teams are looking for solutions for "write once, run on many"
Instant Replay answers a common question amongst development teams - "how did that happen?" - by showing devs a video replay of exactly what a user was doing in the moments leading up to a crash, which means the dev gets an exact set of steps needed to reproduce it.
It looks like webhooks are wedged.. no?
Also, I'm glad to hear that your on-boarding was smooth!
Further, we plan to always have a free tier (for Open Source projects, and for hobby / indie developers).
That said, we'll have a multi-tier pricing offering and plan to announce those specific in the next few months!
At a high level, Buddybuild is a mobile focused continuous integration, continuous deployment and user feedback solution that takes minutes to setup.
We want to help mobile developers build better apps, faster. To do so, we've built an end-to-end system that unlocks a simple workflow: a simple 'git push' to build, deploy, gather feedback and iterate.
As we tried using existing solutions, we felt that there was an unnecessary amount of complexity involved in creating builds, getting those builds to users and then getting meaningful and actionable feedback from those users.
If you're an iOS developer, you'll never have to manage new devices, UDIDs, provisioning profiles or signing identities again. New testers and their devices are automatically provisioned on your behalf through buddybuild.
Further, buddybuild’s deployment service can deploy instantly on every build, every night, or at the push of a button.
By simply taking a screenshot, testers can send graphically annotated screenshots and feedback notes. If your app crashes, buddybuild will trace back and highlight the exact line of source code that caused the crash, tell you which users were affected, and how many times the crash has occurred.
Finally, get a heads up on any potential breaking changes in Xcode. Within 48 hours of the release, buddybuild automatically takes your most recent successful build, builds and runs tests against the latest version of Xcode (including betas!) and emails you the results.
Of course, built in integrations with your favorite services - GitHub, BitBucket, GitLab, Slack, JIRA Pivotal Tracker, Slack, and HipChat.
With buddybuild, you no longer need to cobble together disparate systems to be able to iterate quickly on mobile.
If you want to get a sense for the experience, you checkout http://www.buddybuild.com/demo
Happy to answer any questions people might have!
First, there are many technology companies in Vancouver. Hootsuite, Bench, A Thinking Ape, Clio, Slack, Amazon.. these are just the ones that came to mind. I know authoritatively that each and every single one is looking for senior sales, marketing, engineering and product talent.
Here is a slightly more comprehensive list: http://www.techvibes.com/company-directory/vancouver/tag/sta...
Is the tech ecosystem as big as in SF? No.. but where else is it?
Second, yes.. it is an incredibly expensive place to live. Part of this is due to the large number of homes which are owned by foreigners and are vacant. I'm not aware of a concrete way of measuring this.. but anecdotally, I suspect it's close to 30-40% of the downtown residential core. This is definitely a worrisome issue, but too complex to cover in just a few lines of this post.
Third, be careful not to compare (downtown) Vancouver with it's suburbs. The downtown core is one of the few places in the world where people live and work in the same area... there are a multitude of activities. As many as you'd expect of a city of this size.
Finally, commutes range from 5-30 minutes (and that's going to cover people who walk, bike and take transit).
Vancouver is consistently rated as one of the most livable cities in the world.. but there will definitely need to be changes if we want that to continue to be the case in the long term.
Super valuable feedback. We're in a model at the moment where we want the buying experience to be via the designers. That said, the idea of a 'marketplace' can be found here: http://onceovers.com/providers
We reached out to some of the best designers we know of and asked if they'd be willing to create a 15-20 minute screencast critiquing work... and they were.
We're doing the MVP thing at the moment.. is this useful?