bayan1234

For this reason, I use gpt4 mostly just to start off new functions and then I edit them myself.

It would be awesome if gpt4 could be made to edit the code and therefore, it didn’t have to regenerate it from scratch every time.

The regeneration also chews up a lot of the token limit, so it forgets crucial parts earlier on in the conversation.

Yes exactly. My company has asked AWS if they will be adding support for pgvector for rds but they haven't been able to confirm if that will happen any time soon.

If the vectors are in the same database as the tabular/structured data then text to sql applications of llm's are so much more powerful. The generative models will then be able to form complex queries to find similarity as well as perform aggregation, filtering and joining across datasets. To do this today with a separate dedicated vector db is quite painful.

The hack to solve this is to embed each paragraph in your large corpus. Find paragraphs most similar to the user query using embeddings. Put the paragraphs and the raw user query into a prompt template. Send the final generated prompt to gpt3.

This actually works surprisingly well.

Check out the OpenAI cookbook for examples.

I would subscribe but your website doesn't have a clear demo video of a child using the product to read. It looks very interesting and useful though!

Fivetran, Stitch, Meltano and Airbyte are tools used to extract data from tools such as these into databases.

Census and Hightouch can be used to sync data into these tools from databases.

Forecast package in R is quite useful. Even if you don’t use R, this book by Rob Hyndman is very approachable and easy to follow.

https://otexts.com/fpp2/

Create features for day of week, day of year, month of year, lagged values of y, lagged values of y for each period (eg: 1, 2, 3 weeks and years ago etc). You then predict forward 1 time step at a time.

I think the author isn't talking about changing the product, but the market positioning and how it's communicated to customers.

This looks really cool! I really like the free form chart builder. I work at Simply Wall St and we also make financial data easy to use and understand. We’ve realised that a bigger portion of our effort should be spent on educating individual investors so it looks like we’ve come to the same conclusion as both of you.

If anyone is keen to checkout Simply Wall St (https://www.simplywall.st), we have a 7-day free trial and a free plan for ongoing light usage.

I was initially excited about this as I think it might solve our Redshift pain points and potentially avoid us having to deal with a migration to Snowflake but then I remembered when AWS account managers promised Athena and Spectrum would solve these same problems at a previous company I worked for a few years ago. I'm assuming the developer experience will still be terrible with lots of knobs to tune to actually get any decent cost/performance.

If you have a very stable SQL pipeline that runs frequently during the day then it's probably cheaper to have it on a normal Redshift cluster.

Having used Redshift considerably in the past and experienced significant pain with many aspects of it, I would stay away. There are other much better options, ideally Snowflake.

I read somewhere (a YC video or post I can't find now) that during a pivot, the problem you're working on shouldn't really change, but the solution might change completely.

I interpreted the superordinate goal as the higher level problem you want to solve and dedicate years/decades of your life towards. It might result in multiple startup attempts with entirely different products.

It seems that a superordinate goal which is something you truly believe is worthy is probably well-validated due to deep personal experience. This seems less likely to change easily.

I think one of the less discussed advantages of SQL and reasons why it might be so prevalent in the analytics world is that users don't really need to spend much time to set up their environment. I can send a query to a less technical person with my analysis for them to run. They can put the processed data into a spreadsheet and do whatever they want.

This is exactly my experience. I've found becoming proficient with Pandas to take much more time than other data manipulation libraries like dplyr or Pyspark dataframes. The Pandas way of doing things is just not memorable or intuitive.

Hi, I’m Bayan, author of the post. I would love to hear if you agree/disagree with the definition of a unicorn product analyst or you’ve met anyone who fits the mould. Keen to hear your thoughts.