crawdog

I like the color pallet option you provided showing the primary colors. Adding the colors as facet options would be interesting since you can align each of the colors with the items, then maybe a numeric metric you can use to sort the results by the percent coverage of that color.

Another item to consider is also taking into account how the user might search - "red shoes", then classify the query as "color: red" & "shoes".

The next fun challenge is determining which image to display if you have a variant product (color/size combination for clothing for example). Figuring out which product image to show in the result set requires identifying the primary color in the search.

1. Get to know your TAs and professors. This is important... Knowing how to ask for help is important. Make sure you prepare and walk through the steps you went through to assess any problem and where you are stuck.

2. Enjoy the college experience. Join some clubs. Make friends from different walks of life from you. Take time to socialize.

3. Treat school like your job. If you don’t put effort into it you won’t get anything out of it. Don’t procrastinate if you can avoid it. Good habits now will pay off later in life.

Cobra is great, I created a test project long ago if anyone wants some sample code: https://github.com/rickcrawford/commandline

Jet was purchased for one thing. Marc Lore. The executive team missed their chance to Amazon with Quisdi and were not about to make the same mistake again.

Very interesting. With the speech to text APIs out there would be interesting to further expand to point in time queries of the articles. Similar to the https://podcastsearch.david-smith.org.

Facets would be a nice to have - such as:

Series, Category, Guest, Date

Extra credit:

Speaker diarization and be able to search by individual speakers! If multiple channel feeds were available for the podcasts this would be easier to do... Maybe a search engine for podcasts where you partner with the content creators and give them incentives to tag/provide better feeds?

edit formatting.

Debt financed companies without clear path to revenue are going to be on hold for a while.

Interesting to keep an eye on the CA EDD Warn Act page and see who else shows up:

https://www.edd.ca.gov/Jobs_and_Training/Layoff_Services_WAR...

Learning how to sharpen your knife will go a lot farther than the type of knife you have!

Even hand made/custom sushi knives are < $200 (in Japan)

There's a lot! Sentence detection, parts of speech (POS) detection to name a couple. These can be used to determine key concepts in documents that lack metadata. For example: you could cluster on common phrases to identify relationships in data.

The GSA "just worked". Also having the name recognition for public search did a lot to drive its enterprise popularity. It didn't have a lot of bells/whistles, but it addressed the commodity search problem well.

Agreed relevancy is a problem. PageRank works well for public content but internal search has plenty of relevancy problems. Having little control over this certainly hurt.

Managing on-prem hardware/appliances is a difficult business. I don't fault them for moving to a more scaleable model.

You could throw ThoughtSpot in the mix as well. Interesting to see all the search engines position in the "Insight Engine" space now. Mix of analytics/search, but still very customer specific/custom. Curious how many are successful out of the box.

Also - see the rise in popularity of OpenShift/PCS/PKS - flexible infrastructure is also catching on.

For large (mature) enterprises, I believe at this point it's safe to expect some level of hybrid cloud architecture. I also agree it would be very difficult/impossible to support this for "realtime" indexing.

Interesting to see more players joining the market. You can't walk into a large Enterprise and start your search conversation with "Your developers just build ____". Otherwise customer will want to build it themselves.

The killer feature I haven't seen with many of these solutions is easy, out of the box integration with internal systems (Atlassian Confluence, JIRA, Remedy, SharePoint, FileSystem, Intranet). When you have a SaaS search engine it's difficult to export that data... Even worse to secure it. Ironically, Plumtree Software (bought by BEA -> Oracle) had all of this in their product in 2001. What's old is new again... Those features are prime for a comeback.

I think this is a space where Elastic can do well with an on-prem or managed cloud offering that is "behind the firewall", integrated with customer's environment. Add in term vector search support, ML for document/query understanding, and integration with customer's security model (Active Directory) and it would be compelling.

Court Appointed Special Advocates for Children - Kids in the foster care program often need an advocate and a role model. This organization is a group of amazing volunteers who stand up for those who need our help the most. There are chapters for most cities. https://casaforchildren.org/

Also don't forget your time is a valuable resource. There are many local non-profits and charities that would appreciate 1 hour of your time, even if it is drop in. Local food banks, soup kitchens are easy ways to get into volunteering. It's also a great opportunity to involve your family. If you can coordinate a group of people that helps even more!

I can think of only rare occasions where any status other than 200 is cached. Usually this is an edge case that has to be configured in the caching tier.

"Building its own cloud" = leasing data centers from Equinix or other large data center providers. Financial services have challenges using public cloud due to regulatory and compliance requirements. Most of these challenges are self made - artifacts of moving their teams from "This is how we currently do it" to a shared services model.

Curious if they are all in on providers like Redhat Openshift or Pivotal Cloud Foundry as their PaaS layer.

Does anyone have links to papers on text summarization that you would recommend for research?

Reminds me of another business in the delivery space: https://en.wikipedia.org/wiki/Kozmo.com

Interesting how some of the business models that were not profitable in the early 2000's are resurrected with the on-demand workforce - cutting out the most expensive part.

One more point - if you can run the entire index on one machine it makes deploying and managing SOLR much easier to manage than Elastic. The complexity only grows when you have a distributed system. You can fit a lot on a big box.

Definitely. Product catalog information as a whole doesn't change often. Price and availability does. With retail catalogs you often do a full reindex of the data in your master catalog and then run partials to account for price/availability if you don't do that in realtime using filters. Since the system of record is not the search index, Elastic is often not a good solution here.

Also, relevancy in retail is often influenced by other factors that cannot easily be implemented in Elastic. TFIDF/BM25 search is available in both platforms, but you may also weigh in other factors such as relationships with the vendor, stock on hand, or other ML techniques that are more easily implemented in SOLR.

thanks for catching. updated.

corrected!

Very much so. For retail/catalog search SOLR dominates. There's a lot more customization available for relevancy/ranking OOB than Elastic. Drawbacks are managing indexing - SOLR cloud is much harder to manage.

For commodity search workloads (general retrieval/faceting) Elastic does a fine job. It scales well and there is good documentation and support.

Lucene is the core engine behind both of these solutions.

For fun, lets look at the large Enterprise acquisitions over the years:

* Verity - bought by Autonomy

* Fast - bought by Microsoft (Also known as the Enron of Norway...)

* Autonomy - bought by HP (Look at the backstory on this deal!)

* Endeca - bought by Oracle

* Vivisimo - bought by Oracle

* Google - GSA (now Google Cloud Search, hosted solution)

Next, follow the path of online acquisitions:

* IndexTank - bought by LinkedIn

* Swiftype - bought by Elastic

There's a number of interesting independent players still. Coveo plays in the Enterprise space, but it's a hard market. Algolia is doing great in the commodity online search space and seems to be growing well.

This is an area I think is open to more competition. Especially with AI/ML technologies available around Document Understanding - the Enterprise market is open for a good on-prem upstart to really take off.

Ping me offline if you have additional questions - spent almost 20 years in the space and ran a search company of my own.

The number of auctions in the early 2000s was staggering! Furniture and art were the two cheapest items to buy...

Port Costa, CA used to have a train ferry in the gold rush era. If you ever have a chance to visit it's a cool little town. Hidden just south of Martinez in the Bay Area.

https://en.wikipedia.org/wiki/Solano_(ferry)

Heres an old project I wrote with some sample code for the C binding library. I tried the go versions and always found the c binding faster.

https://github.com/rickcrawford/go-lua-test

As long as it isn't PHP you should be ok...

If you are looking for public spaces to work - the lobby of LinkedIn at 222 2nd is very nice and has access to free wifi/coffee.

Adding to places to visit: * SF Museum of Modern Art * Hike Point Reyes - get some oysters in Tomales Bay on the way back * SF Cable Car museum - Free!

El Farolito for the torta. Cancun for the burrito!

Take a look at http://runnable.com