cschiller

By default, the SDKs use our API endpoints, where we run a combination of models to maximize accuracy and reliability. This also enables us to provide logging with screenshots and reasoning to help with debugging.

That said, we're currently experimenting with a few customers who run our tooling against their own hosted models. While it's not publicly available yet, we might introduce that option going forward.

Would love to hear more about your use case, if a self-hosted setup is relevant or just the use of your own LLM tokens?

Yes, you can use our SDKs to run it locally on Simulators, Emulators, and real devices. We also support popular third-party device farms via the WebDriver protocol.

Hi Jztan, glad you're exploring this space! I'm the co-founder of MobileBoost, and I'd love to introduce our product, GPT Driver (https://www.mobileboost.io/).

We started two years ago with an AI-native approach, which is particularly useful for handling dynamic flows, hard-to-locate UI elements, and testing across multiple platforms and languages. Our main objective is to reduce test maintenance effort.

Duolingo recently shared their experience adopting our tooling: https://blog.duolingo.com/reduced-regression-testing/

We offer: a Web Studio – A no-setup-required platform with all tooling preconfigured. SDKs – Directly integrate with existing test suites (Appium, XCUI, Espresso).

Happy to answer any questions!

Good call! The timing was actually a coincidence, but not unexpected. OpenAI had already announced their plans to work on a desktop agent, so it was only a matter of time.

From our tests, even the latest model snapshots aren't yet reliable enough in positional accuracy. That's why we still rely on augmenting them with specialized object detection models. As foundational models continue to improve, we believe our QA suite - covering test case management, reporting, agent orchestration, and infrastructure - will become more relevant for the end user. Exciting times ahead!

Thank you! Sonnet 3.5 is indeed a powerful model, and we're actually using it. However, even with the latest version, there are still some limitations affecting our specific use case. For instance, the model struggles to accurately recognize semi-overlaid areas, such as popups that block interactions, and it has trouble consistently detecting when UI elements are in a disabled state.

To address these issues, we enhance the models with our own custom logic and specialized models, which helps us achieve more reliable results.

Looking forward, we expect our QA Studio to become even more powerful as we integrate tools like test management, reporting, and infrastructure, especially as models improve. We're excited about the possibilities ahead!

Thanks for sharing your experience! Completely agree - there's often a huge gap between the perception that testing is "solved" and the reality of manual QA still being necessary, even for core features. We recently had a call with one of the largest US mobile teams and were surprised to learn they're still doing extensive manual testing because some use cases remain uncovered by traditional tools. It's definitely not as "solved" as many might think.

Thanks for your thoughtful response! Agree that digging into the root cause of a failure, especially in complex microservice setups, can be incredibly time-consuming.

Regarding writing robust e2e tests, I think it really depends on the team's experience and the organization’s setup. We’ve found that in some organizations—particularly those with large, fast-moving engineering teams—test creation and maintenance can still be a bottleneck due to the flakiness of their e2e tests.

For example, we’ve seen an e-commerce team with 150+ mobile engineers struggle to keep their functional tests up-to-date while the company was running copy and marketing experiments. Another team in the food delivery space faced issues where unrelated changes in webviews caused their e2e tests to fail, making it impossible to run tests in a production-like system.

Our goal is to help free up that time so that teams can focus on solving bigger challenges, like the debugging problems you’ve mentioned.

One of our customers recently compared GPTD with Maestro’s Robin (formerly App Quality CoPilot). Their mobile platform engineering manager highlighted three key reasons for choosing us: lack of frustration, ease of implementation, and reliability.

To be more concrete their words were: - “What you define, you can tweak, touch the detail, and customize, saving you time.” - “You don’t entirely rely on AI. You stay involved, avoiding misinterpretations by AI.” - “Flexibility to refine, by using templates and triggering partial tests, features that come from real-world experience. This speeds up the process significantly.”

Our understanding is that because we launched the first version of GPT Driver in April 2023, we’ve built it in an “AI-native” way, while other tools are simply adding AI-based features on top. We worked closely with leading mobile teams, including Duolingo, to ensure we stay as aligned as possible with real-world challenges.

While our focus is on mobile, GPT Driver also works effectively on web platforms.

I agree that it can seem counterintuitive at first to apply LLM solutions to testing. However, in end-to-end testing, we’ve found that introducing a level of flexibility can actually be beneficial.

Take, for example, scenarios involving social logins or payments where external webviews are opened. These often trigger cookie consent forms or other unexpected elements, which the app developer has limited control over. The complexity increases when these elements have unstable identifiers or frequently changing attributes. In such cases, even though the core functionality (e.g., logging in) works as expected, traditional test automation often fails, requiring constant maintenance.

The key, as to other comments, is ensuring the solution is good at distinguishing between meaningful test issues and non issues.

I would say that mobile apps are still the primary format for launching new consumer services, incl. new apps like ChatGPT and many others. However we’ve observed that teams are expected to do more with less—delivering high-quality products while ensuring compliance, often with the same or even smaller team sizes. This is why we focus on minimizing the engineering burden, particularly when it comes to repetitive tasks like regression testing, which can be especially painful to maintain in the mobile ecosystem due to use of third-party integrations (authentication, payments, etc.).

Yes, great point! We have an 'Assistant' feature where you can perform the flow on the device, and we automatically generate the test case as you navigate the app. As you mentioned, it’s a great starting point to quickly automate the functional flow. Afterwards, you can add more detailed assertions as needed. Technically we do this by using both the UI hierarchy from the app as well as vision models to generate the test prompt.

If you are interested in getting weekly recipe ideas tailored to your preferences, easy to follow step by step cooking instructions, and a handy shopping list export / cart transfer to Amazon Fresh/Whole Foods Market feel free to check out Kitchenful (https://www.producthunt.com/products/kitchenful#kitchenful).

It's an app that I have built with my team over the last few months to streamline the cooking at home process.

Thank you for it being so well said!

The beauty of it is that everyone is likely to face or notice problems in their own life which offer the chance to be solved. And usually one is not the only one facing it :)

Hopefully this inspires someone to solve a problem which nags them and which they can get passionate about.

We are excited about the opportunity to work on this problem. And are grateful for the constructive feedback we are receiving from the HN community.

Thank you for your structured and helpful feedback!

Let me try to cover some of the points:

- We have a small team of passionate foodies producing their own recipes and reviewing/curating weekly recommendations for our users. The team has an international background (incl. US/EU/Asia), which helps to tap into various experiences when it comes to generating the weekly menus. They have worked at some of the most popular European recipe apps. Feel free to check out our Instagram account to explore some of the recipes they have produced so far (https://www.instagram.com/kitchenful/)

- We provide our users with a personal menu each week. In addition there is the option to choose from standard categories and your own favorite recipes. Currently it’s not possible to import your own recipes, but we have it in the backlog. So far this hasn’t been a blocker for our users to use our service on a weekly basis.

- As of now we don’t have a feature to export the recipes. Mostly because so far we didn’t get that request from our users. But I see the peace of mind value in it as a user and will look into this.

- We are building our frontend clients with Flutter. So we look forward to supporting many platforms (iOS, Android, Web, Mac, Linux, ...) soon, stay tuned.

- Currently we are working with custom ranking and rotation algos for the recipe collation. But ML definitely on the horizon as we increase the number of data points :)

Happy to ping you once we have implemented a bit more features on the multi-platform and recipe import/export side. Excited to get your early feedback then.

Thanks for the feedback! Very helpful for us to think about how we can showcase our service before the sign-up event and remove the uncertainty related to location.

Do you mind me asking: in terms of food, what would be most important to you before signup? Is it to see the quality of the recipes, or rather a test if we can make good recommendations based on your specific preferences and needs?

Thanks for the kind words! And yes, indeed we feel that the combination with delivery services makes a significant difference. By improving the inspiration experience and doing the related shopping on your behalf, we’re trying to deliver a smooth and successful cooking and dinner experience without the hassle. We believe that this is critical for a service like ours.

And in addition, as you pointed out, with each order we learn from the integrated shopping about the user’s preferences which enables us to continuously improve the service over time.

Thanks for the feedback and apologies for any confusion! The lack of clarity about the next step after signing up is our bad, we are currently working on making the process clearer. Happy to see Berlin users here!

Platejoy and plantoeat are great products!

We focus on two things which I believe set us apart: (1) creating personal suggestions each week based on your preferences, diets and goals, which are currently reviewed and curated by real chefs (2) taking care of placing the order for you with your favorite grocer. Meaning the order process for you is seamless. You simply choose the meals you would like to cook, and any other specific products (like beverages, snacks, etc.) and place your order with 1 click.

We’ve found that this combination solves a bunch of problems existing solutions still have from a user perspective.

Thanks for the point about the auto-renew. I can personally relate to this. We decided to go for the auto-renew on a monthly basis, meaning our users can cancel anytime with limited downside and without the need to commit to long time spans. But we’ll definitely take another look at the opt-in models, thanks for the hint!

Looks like a great software product for household management!

What we learned is that when it comes to dinners, many decision makers in the household struggle with making the decision every week about what to eat for the next few days. The randomized recipe button seems to go into that direction. However making relevant suggestions is what we learned so far to be a key in the experience (e.g. considering the current season, what was for dinner in the past week, what is similar to a favorite dish but with a new twist, etc.)

From our existing users we get the feedback that a personalized menu each week and a much more convenient way to order their groceries saves them time and mental energy. For many worth the price. Feel free to give it a try and let me know how it works.

Main difference is the strong focus on your preferences, diets, and goals. We use a combination of our own produced recipes, as well as recipes from food creators we work with. This allows us to tap into a wide range of different meals from the start and tailor the weekly recommendations to each individual.

On top of the personalization, we also have a different order process in which we take care of finding the best substitutes in case something is out of stock, and then provide you with personalized cooking instructions after the order.

Feel free to give it a try and let me know how it works for you.

True, the picking quality is a big factor for the experience. Especially when it’s about shopping for ingredients for a meal. We try to improve this by placing the order on behalf of the customer, so we can optimize the product selection for the cooking experience and have the option to guide the pickers with comments/chat towards the most appropriate products or suitable substitutes if something is out of stock.

Besides that we are considering to give the user an option to simply export the optimized shopping list, so it can be used for offline shopping. We got this a few times as a feature request from users who like to shop the ingredients themselves, but still like a service which provides them with personalized suggestions each week and generates a combined shopping list.

Sorry for the confusion with the UK logos. Unfortunately we’re not operating in the UK yet but are definitely looking into it.

Good spot! Will amend that asap. Thanks for the kind words though! Look forward to serving you.

Thank you! Looking forward to serving you in the near future.

Thanks! Generating nutritionally complete, minimalist diets sounds super interesting. Feel free to link it here or write to me at cs@kitchenful.com so we can take a look.

Great question! So far we’ve experimented with the quick commerce players mostly to cover for out of stock items from the bigger grocers. E.g. if the grocer was out of tomatoes at the point of delivery, we offer our users the option to have the out of stock item delivered by a quick commerce service around the same time of the main delivery.

The majority of our current users seem to prefer one big grocery delivery per week. Because the benefit of having the planning and shopping done a few days in advance means that it prevents decision making on the other days.

Having said that, we believe that shorter times from planning to actual delivery are still key in the user experience. So same day delivery and pickup options by the grocers help to improve the overall experience.

This sounds like a great hack! Personally we faced a kind of cold start problem with such a diary. So what we’re trying now with Kitchenful is to recommend new recipes that are similar to a few favorite dishes of our users, which we ask about in the onboarding process.

We like the idea of a “Future” list and are already thinking about a bookmarking / recipe import feature. How do you save future ideas in your app? (urls?)

stay tuned :)

Thanks for the idea! We used https://ipapi.com/ at the start to render different page content based on the country of the visitor. But like the idea of making this clearer on the homepage in general, because an ip address lookup might not be accurate and hard if there is no way for the user to influence / change the language/location.

Thanks! Having less packaging waste is actually one of the points our early customers who’d used a Meal Kit service before like about Kitchenful.