The standard library has type hints in the "typeshed" github repo. Please do not submit PRs to cpython to add type hints (I made this error before too:))
424 karma · joined September 7, 2022
Former SWE.
As somebody who finds it harder to gain weight, I still recognise that most have the opposite problem and understand that most content will be tailored to the average Joe, which is fine.
Thinking of the stupidest way we can get it done and then ruthlessly refactoring until the resulting algorithm / system is as simple as possible.
Edit: point 1 comes from A Philosophy of Software Design. One of my fave programming books of all time.
What does this mean? Sorry, I don't speak French..
Are you against animal ownership in general or just dogs?
Did management also forego raises alongside devs?
Whether or not the company is behaving rationally is the wrong question to ask I think, all you can change is how you manage yourself.
Solving them is the hard part;)
Criticism would be moderated through the journal editor, as with typical review comments.
I've used gensim's one and it seems great for practical use but I wanted to build an understanding of the underlying algorithm, so built my own from scratch. I was hoping others might learn from it, but it's not really a practical approach to constructing w2v embeddings.
Here's an example output:
Array Shapes:
==================================================
attention1 : Tensor : (8, 8921, 251)
m : Tensor : (8, 8921, 50)
ŷ0 : Tensor : (8, 8921, 50)
ŷ1 : Tensor : (8, 8921)
ŷ2 : Tensor : (8, 8921)
random_np_arr : ndarray: (2, 3)
print(f"\nArray Shapes:\n{'':=>50}"); [print(f"{k:<20}: {str(type(v).__name__):<7}:", tuple(v.shape)) for k, v in locals().items() if hasattr(v, "shape") and not hasattr(getattr(v, "shape", None), "__call__")] # Exclude the 'numpy' import, the `shape` func. isn't relevant
For example in the sentence:
> Usually titles are more relevant to a document then a random sentence or word in the middle of it
If 'document' is the current target word, we select ["relevant", "to", "a"] and ["then, "a", "random"] as our left and right contexts. These context words and our target word, ["document"] form our 'positive' examples that we want to move closer together in the embedding space.
Position in the text is not used in this algorithm. You're right about titles being more relevant to the meaning of a document but the document isn't important: it's just a resource that we're using to learn which words occur together (and thus are semantically related). The algorithm steps through every single word as a target and thus will inevitably utilize words that were titles just as much as words in any other part of the text:)
Hope that helps!
The lines are processed independently as they are separate sentences. One line may be from one source, and the next from a completely different source. The problem with simply concatenating them is that the target words on the boundaries between sentences can end up with context words from unrelated sentences.
>I liked seeing the 'η' - I haven't yet used non-ASCII in my code!
Then you may enjoy this april fool's python issue I made https://github.com/python/cpython/issues/103172
Yeah, I like using utf-8 chars for mathematical symbols, it makes my brain hurt a little less when mapping between the two! I also like using ŷ for predictions in ML contexts as that's the canonical symbol used in the literature.
In practice, as you intuit, contextual embeddings are in fact useful for obtaining more precise word vectors. The example given in[0] is BERT's embeddings, though I would imagine that most NLP applications nowadays all use contextual embeddings since they're a strict improvement over static embeddings.
[0]. https://web.stanford.edu/~jurafsky/slp3/6.pdf (search for 'static embeddings')