Dask is popular for this purpose. No memory compression, but supports disk spillage as well as distributed dataframes.
50 karma · joined September 27, 2011
What is the training speed of this network? Computation seems to scale at N^2 rather than N.
I'm now working in the ML/AI research division of big-4, all I needed (and all hiring teams needed to see on my resume) were a couple of these classes (with good internal performance ratings).
Having said that, my employer paid for these classes. No way I would pay this price out of pocket. There are probably much cheaper ways to get the same knowledge.
You actually study, partner with, and test along-side Stanford MS/PhD students. Not to mention the accountability factor, since you paid $5k for the class you better take it seriously.
That indeed does not exist, but if you do well on these, you are probably pretty eligible for their part-time MS program.
In any class of reasonable difficulty/usefulness, there will be students who do great, ones that do okay, and ones that struggle. What we should aim to do as the very first intro course is to help as many of those people from all of those groups to do as well as possible in their future classes/careers.
For the great students, I actually believe we did them a disservice. CS61A in scheme was truly beautiful. It was amazing, given how simple the language was, how much you could achieve. All pieces of the puzzle fit together perfectly. Trying to do the course in python, on the other hand, we had to compromise a little bit. We didn't want to just teach the exact same content in a different language. We wanted to teach the "Pythononic" way of doing things. To give a few examples: - The first 1/3 of CS61A in scheme had no mutation. That is almost impossible in Python to do, and is not the way the language is used in industry. I personally believe functional programming and immutability is The-Way to program, so I see this as a huge loss. - We teach recursive lists (linked lists) in order to teach recursion, but I feel like having both the default Python mutable lists (which are array lists) and linked lists is confusing to students, since we do not teach their performance tradeoffs (that is in 61B). - Mutual recursion was almost useless to teach. Where it was most commonly used in Scheme, you achieve the similar thing using list comprehensions in Python (the language feature essentially took away your need to construct the list in a recursive fashion). - The very interesting bits of the old class about metacircular evaluators, we simply could not do in Python. - Python also has tons of quirks (magic methods, for instance) and lost some of the elegance we saw in scheme.
But for the okay and struggling students - 70-80 percentile and below, the students who understand most of the material, but perhaps fail to grasp some of the more complex concepts - I think Python was the correct way to go for a few reasons: - It is a much more commonly used language. It sets you up to go for internships, research, hackathons, whatever you please, straight from what you learned in class. - The language is way more similar to Java, which allows students to translate what they learned easier to the next course in the series, CS61B. - 80% of the material hasn't changed but you've gained the above benefits. Remember, we are talking about students who probably would have struggled to grasp those last 20% anyway (maybe they are completely new to programming and they already have their hands full with the 80% to begin with).
I think therein lies the difference in opinion. If you took CS61A in scheme and understood it very well, then you don't understand the change. If you understood the intricacies of what was taught in CS61A, you will find it very easy to generalize those concepts to new languages - and new concepts. In fact, you will find that much of what you learn in the rest of your undergrad career will be "CS61A review". However, for students who may not have had the CS maturity or time to grasp all the concepts, they find that the Python course is much more useful.
I am also curious about how the NLP/ML parts are implemented, as it's claimed by the README on github. Briefly scanned the code but didn't really spot it.
OpenFile returns type File from the signature. http://golang.org/pkg/os/#OpenFile
Scanning through the methods (or doing a cmd + f search for "disk") has this method http://golang.org/pkg/os/#File.Sync
Method description says "Sync commits the current contents of the file to stable storage. Typically, this means flushing the file system's in-memory copy of recently written data to disk." Isn't that what you want?
Was it as clear as the C# example? Probably not. But I don't think this was terribly hard to find. Code examples will come in time, as the language matures.
IMO whether to put code examples into the documentation is really a choice by the language maintainers. Lots of code examples can also clutter the docs. I prefer to just have the docs tell me what each function does. If I really get confused, I can google for examples. Java also doesn't really have code examples right in the documentation, but they aren't hard to find because it's been around for a long time. For example, Googling for "java code example how to append to a file" and click on the first link.
Dynamic properties of languages should only be used from the perspective of saving time - it's less work to make changes to code because you don't need to conform to a type system. I feel like more often than not it's used as an excuse to write sloppy code. You simply cannot write good code without keeping your types and function signatures correctly.
+ Syntastic [https://github.com/scrooloose/syntastic/] eclipse-like real-time linter. Gives me compile errors and warnings.
+ Tag bar [http://majutsushi.github.com/tagbar/] navigate tags in the code
+ Not as essential, but EasyMotion is pretty awesome [https://github.com/Lokaltog/vim-easymotion]
On a completely different note, though, I wonder what the range on these things is. Could have excellent applications to robotics, I hope they don't completely close the vision outputs behind their own gesture APIs or something...
We are still striving to teach the same material, just in a different language. This class is not at all a class about Python, just as the old course was not a class about Scheme. The course content is almost the same: functional programming, data abstraction and data structures, OOP, interpreters, parallel and distributed computing, declarative programming.
We have unfortunately had to sacrifice some great aspects of the original class, eg. the metacircular evaluator. The hope is that in exchange we have a more natural OOP system, better support for parallel and network computing packages, and much more modern implementations of distributed computing.
[disclaimer: these are my own opinions, I do not speak for the department]
I took this class while it was still taught in Scheme, and I loved it. Now I'm teaching it in Python. I am also sad to see Scheme go, but I think we gained as much as we have lost in our switch to Python. For example, I argue that Python dictionaries are more intuitive to use than the old "association lists" implementation in Scheme (we still taught the implementation). Concepts like MapReduce and concurrency that we cover later on in the course are also cleaner and more elegant than the Scheme implementation. The above-the-line OO syntax is also much, much easier to use.
We are still covering interpreters as our last unit. It will be for a "calculator" language, with conditionals and assignments. We are hoping to cover a lot of the same concepts, but we decided that a metacircular interpreter is obviously too difficult. We are keeping project 4 the same (Logo interpreter) but we have ported it to Python and wrote some new questions. Personally, I think the OO-centric interpreter is an improvement over the old Logo interpreter in Scheme (we can have Environment objects that now contain Frame objects, for example. Before, environments were just a list of association lists).