> otherwise iOS would be an awful platform for developers and by extension the users of Apples platform
This seems to imply that iOS is a a good platform for developers, and by extension, the users of Apple's platform.
2,662 karma · joined October 31, 2016
This seems to imply that iOS is a a good platform for developers, and by extension, the users of Apple's platform.
I was working for a startup that first had the strategy of testing everything, at a point when we didn’t even start working on the product. We spent a horrible ammount of time getting the test system up and running (microservices, browser add-ons), with the UI testing being the most challenging.
But then they also wanted to "move fast and break things", which we did, so we spent an annoying ammount of time fixing breaking tests. This is when the “let’s just delete the test” expression started popping up.
So we ended up with an unmaintained testing system that no one cared about anymore, keeping our build on red because we always had the testing system in the backlog so it was going to be done at one point.
Then, the quality of the codebase started degrading to the point where we’d wake up with features that have been broken for a while and no one noticed it, even though each individual developer was testing their own flows. We had no one / nothing testing the entire system.
At this point, I suggested we hire 1 – 2 manual testers, as our testing strategies are obviously failing, the product is suffering in terms of quality and we could get them for relatively cheap compared to dev-time. I’ve had great success working with manual testers for very complex products with real world repercussions, compared to this tiny start up in dev tooling.
They refused. So they decided we’d do cross functional testing and then test the entire system whenever we’d do a merge. So developer velocity fell from a cliff because we became the manual testers. We still had no customers at this point. Runway got shorter and shorter. And the start-up became a statistic.
Close to engineering, what about the SCRUM masters? The testers. What about the product owners? What about devops? Further from us, what about the people signing up on our vacation? Or the ones signing up on our daily budgets when traveling, or hell, even the ones we interview with.
In my closest group of friends (we're all seniors in our domains, and very honest with eachother), I find that only the construction worker's job should be safe. And compared to myself and the devops guy, most others have what they themselves describe as trivial and bullshit jobs. Join a meeting, do some paper pushing, some document signing, a little coffee and the day is over by 1PM.
Am I seeing all these AIs replacing programming because I'm on a board where maybe a lot of us are programmers? Is it the same for other roles? Wouldn't it make more sense to have the interview process automated by LLMs if they're capable of building great software, before we replace those hired?
I'm very confused by all the hype when matched with my experience of using LLMs daily for the past years.
I'm talking about 4-turbo, 4-turbo preview and self hosted LLama2. What in God's name is not SOTA about this?
I hack on them till I get something sort of satisfying.
> You'd be surprised how much better your output is using prompting techniques tailored for your goal.
The biggest problem I encounter is context length, not necessarily the output for small inputs. It starts forgetting very fast, whether it's Claude, GPT+ or other self hosted models I've tried.
I also have two crypto-bro friends that are hyping it up without having anything to show for it. Which is why I'm sort of complaining about they hype surrounding it. I agree with your post to a large extent. This is not production ready technology. Maybe tomorrow.
Sometimes? But I don't go to McDonalds for the loss function between the picture and actual product. I go for the fast food and good taste (YMMV).
> If you go in with a healthy dose of cynicism IMO LLMs can impress.
I use them everyday in one way or another. But they're not replacing me coding today. Maybe tomorrow. And I go in with a healthy dose of optimism when I say this.
> I’d call it a better google search and autocomplete on steroids.
Sure, but this particular discussion is not about its ability to replace Google Search and / or Autocomplete.
I find it horrible at replacing library documentation
> I feel like many of this things require asking the right questions, which assumes certain level of experience. But once you reach this level, it’s an extremely valuable assistant.
I've been using LLM products since incipience. I use them in my daily work life. It's a bit tiring hearing this 'right questions', 'level of experience' and 'reach this level'. Can you share anything concrete that you achieved with ChatGPT that would blow my mind?
I keep hearing this 'you need to ask the right kind of questions bro' from people that never build a single product in their life, and it makes me question my ability to interact with LLM but I never see anything concrete.
I agree. I have it active on VSCode and enjoy it. It has introduced subtle bugs but the souped up autocomplete is nice.
> 2) copilot can help me kickstart writing some complex functions starting from a comment where I tell it what is the input and expected output. Is the implementation always perfect or bug free? No. But in general I just need to review and check rather than come up with the instruction entirely.
I don't find it very useful for anything non trivial. If anything I found it more useful for generating milestones and tasks for a product, than even making a moderately complex input -> output without me having to check it in a way that annoys me.
> 3) copilot chat helps me a lot in those situations where I would've googled to find how to do this or that and spent a lot of time with irrelevant or outdated search results
I find I don't use copilot chat, almost at all. Nowadays I prefer to go to Gemini and throw in my question.
> 4) I have found use cases for LLMs in production. I had lots of unformatted plain text that I wanted to transform in markdown. All I needed to do is to provide few examples and it did everything on its own. No need to implement complex parsers, but make a query to OpenAI with the prompt and context. Few euros per month in OpenAI credits is still insanely cheaper than paying tons of money in writing and maintaining software by humans for that use case.
This is mostly what I'm using it for in this current project. It does it job nicely but it's very far away from replacing myself as a programmer. It's more like a `fn:magic(text) -> nicer text`. This is a good use case. But it's a tool, not a replacement.
> 5) It helps me tremendously when trying to learn new programming languages or remembering some APIs. Writing CSS selectors is actually a very good example. But I don't feed it an entire HTML as you do, I literally tell him "how do I target the odd numbered list elements that are descendants of .foo-bar for this specific media query". Not sure why would you need to feed it an entire HTML.
Because I get random websites with complex markup, and more often than not every page has its unique structure. I can't just say give me `.foo-bar` because `.foo-bar` might not exist. Which is where the manual process comes in. Currently, I'm using hand crafted queries that get fed into GPT / Claude / LLama, but the actual query is what I wanted it to do.
> 6) LLMs have been extremely useful to generate images and icons for an entire frontend application I wrote
I'm very curious how this behaves in different resolutions. There's a reason vector graphics are a thing. I've used it for this purpose before but it doesn't compare to vectorial formats.
> 7) I instruct him to write and think about test cases about my code. And it does and writes the code and tests. Often thinks about test cases I would've never thought of and catches nice bugs.
What is the context size of your code? It works for trivial snippets but as soon as the system is a bit more complex, I find that it becomes irellevant fairly fast.
> The promise of it writing anything but simple boilerplate, I find it ridiculous because there's way too much nuance in our products, business, devices, systems that you need to follow and work on.
> But as a helper? It's terrific.
> I'm 100% sure that people not using these tools are effectively limiting themselves and their productivity.
Totally agree. But I'm not complaining about its usefulness. I'm a paying user of LLM systems. I use them almost every day. They're part of my products. But this particular hype about it replacing ... me. I don't buy. Yet. It could come tomorrow and I'd be happier for it.
Just yesterday I tried to feed it a simple HTML page to extract a selector, I tried it with GPT-4-turbo, I tried it with Claude, I tried it with Groq, I tried it with a local LLama2 model with 128k context window. None of them worked. This is a task that while annoying, I do in about 10 seconds.
Sure, I'm open to the possibility that in the next 2 - 3 days up to a couple of years, I'll no longer do manual coding. But honestly. After so much hype, I'm starting to grow a bit irritated with the hype.
Just give me a product that works as advertised and I'll throw money your way because I have a lot more ideas than I have code throughoutput!
There's body shops in Eastern Europe that pay more for junior frontend developers...
It's just horrible to investigate a customer issue and is a frustrating experience both for the devs and the customers.
"Hello, <customer>, do you mind doing the exact same operation you were complaining about right now?", then it's shifting through microservice logging hell.
On one hand, I have a backoffice that gives access to the staff to everything related to the customers, but on the other hand I'm supposed to believe that being able to 'impersonate' a customer is this highway to hell? As long as an audit log is held about who did what, this can go a long way forward.
Are you Series A? B? A decacorn? Sure, implement whatever distributed tracing dashboard analytics data lake serverless microservice monitoring solution you can dream of. But this is good advice for early stage.
Also most companies have a backoffice that should provide access to the information and possibility to apply actions to the state. So just giving access to this information in a more limited way that mimicks the customer's view is a no brainer for early-stage.
* Small company; two-sided marketplace with some hundreds of thousands on one side, and some tens of thousands on the other, so rather small scale.
Are there many products that reach such sizes without achieving Product Market Fit (PMF)? I feel like after this step is achieved, QA becomes pivotal and involves a great combination of manual and automated procedures. So I agree with you in this regard.
But going back to my initial assumption. I think starting a fresh company without PMF and spending a lot on QA until that is achieved, might not be the best approach.
For me it's a no brainer, if I were CEO / CTO, until product-market-fit is achieved and exponential growth is visible, I'd just outsource Q&A and that's that.