IanOzsvald

Developing more than one revenue stream is likely to be a more resilient strategy than hoping that Google is happy to keep paying a competitor browser to stay in business

I've recently written a related library - given a DataFrame it'll run sklearn's RandomForest to check which columns predict other columns. The goal is to learn which relationships exist within a DataFrame. Typically in the exploratory process in machine learning we want to learn how the data holds together - this tool helps with that discovery exercise. It'll auto-LabelEncode text and allows classification or regression. There are two example Notebooks (Titanic & Boston) to show what it is doing. Correlations (Pearson, Spearman, Kendall) can also be calculated. The RandomForest result can show non-linear relationships that aren't exposed by correlations. https://github.com/ianozsvald/discover_feature_relationships

tensorflow looks to work fine with Python 3.4+ https://github.com/tensorflow/tensorflow

I've just installed it using conda in Python 3.6 just fine.

A few weeks back I surveyed my 3,500 members at PyDataLondon (London-based Python data scientists) about their Python version usage. At work 60% use Python 2.7, 33% use Python 3.4+. At home 44% use Python 2.7, 49% use Python 3.4+. I'm predicting 50% at-work usage of Python 3.4+ this time next year for my London community: http://ianozsvald.com/2016/06/20/results-for-which-version-o...

Note - I'm the co-chair for PyDataLondon (and the local conference series). I regularly poll my monthly meetup audience to see how many folk upgrade each month. Normally 1-10 per month stick their hand up, nobody sticks their hand up if I ask "how many downgraded from Python 3 this month?".

Some years ago (6 or so, from memory) I realised I'd been consulting in a role for too long and gone stale and had lost my creativity (not burned out, but approaching that state). I had just enough money to survive for 6 months so I stopped work, chilled and then started to play with tech again. I returned to consulting after 6 months and everything immediately picked up again.

Years before (11 years ago I think) I left a start-up completely burned out, having just bought an apartment, with no spare money (all my cash went into the purchase). The start-up had run out of salary, I was angry and wasn't in a clear space. I bailed the UK and walked around Morocco for a month on my own (I got very bored of my own company), on the very last of my money. When I returned, very refreshed, I started my consulting career. That break was essential to let myself recover. It surprised me how much the rest of the world hadn't changed in that month, whilst my outlook was totally different (happy, positive and ready for the challenges ahead).

To add some evidence - I co-chair PyDataLondon (3,000 members, UK's largest Python u/g, UK's most active data science group). I survey our members, our monthly attending group are 40% PhD, 40% MSc, 20% other, few have 5-10 yrs industry experience, the majority have 2-4 years. I'd argue that you need at least a relevant MSc + a couple of years experience to begin to talk of being a data scientist/AI engineer. Coming through data engineering in support of data science is a great route to get practical experience where there's a lot of job demand, at least in London.

+1 that conda solves "most of our packaging problems" in the science/data science world (I've lived within it for 1.5 years, it really simplifies things). I use conda on Linux, Mac and Windows for r&d and deployment for tools including numpy, sklearn, matplotlib along with flask, pymongo etc.

Hey fijal. Agreed that the survey has a self-selecting audience, I'd also argue that they're the more forward-thinking folk rather than jobbing background users.

Back in April 2013 (the last time I saw python.org download stats - where did they go?!) I wrote a blog post noting that fresh downloads of Windows Python 3.3 were greater than downloads of Windows Python 2.7, for 3 months running. Windows is useful as Python isn't bundled (unlike e.g. Linux and Mac). I presume this trend has continued but have no firm evidence either way: http://ianozsvald.com/2013/04/15/more-python-3-3-downloads-t...

What do the PyPI stats say?

HotPy is Mark Shannon's research project, he's not making a 'new better Python for the masses', he's using it to test his assumptions about ways CPython could be improved. Ask him about it if you bump into him at a python conference!

The recent Py2 vs Py3 survey over Christmas suggests that approx. 32% of respondents write in Python 3 (increasing on last year), 68% write in Python 2 (decreasing on last year). Py3 usage is up approx. 12% on last year's survey. For personal projects Py2 and Py3 have roughly equal popularity: http://www.randalolson.com/2015/01/30/python-usage-survey-20...

At monthly PyDataLondon meetings I remind the audience to switch to Py3 (a few do each month) as Py2's sunset date is less than 5 years away now.

I use a Dell E6420, as a data scientist I kept breaking the default 8GB so upgraded to 16GB which is almost always fine. I won't go >8GB most days but when I do, I need it (else I lose hours trying to partition data and think about ways to subselect). 16GB for data science seems a sensible minimum for the folk I know. I'm on Linux Mint 17 (Ubuntu 14.04) + Python.

Glad you enjoyed it :-) If you have a moment, leaving a review (e.g. on Amazon) would be most appreciated (there's a dearth of views as it is a bit of a niche subject!)

Much obliged (assuming that's directed at me!) - Radim's started with some rather nice talks too :-)

Don't forget that the higher level functionality (e.g. the scikit-learn routines Radim uses) are typically wrappers for underlying C/Fortran routines and they're the real bottleneck. The relatively few lines of VM'd Python are 'slow' compared to e.g. C but aren't the bottleneck.

The win with Python (and other dynamic languages) is that you can experiment quickly with ideas when you're formulating a solution, that's a big part of exploratory data science.

If you're curious about high-speed work in Python - Radim did a blog series on how he sped up word2vec to be faster than Google's original C code: http://radimrehurek.com/2013/09/deep-learning-with-word2vec-...

I'll also note [self promo!] that I wrote on book on High Performance Python, if that's your cup of tea (and Radim wrote a section in it): http://shop.oreilly.com/product/0636920028963.do

Consider Wikipedia->DBpedia->Wikidata->Wikipedia. First we had semi-structured human-readable information expressed using mediawiki's markup which was hard to parse automatically. Next the DBpedia project (and YAGO) created tools to parse mediawiki and extract facts as triples (e.g. "this thing" "is-a" "company"), they encountered many alternate ways of expressing the same information (e.g. date variants, weights and measures).

Now the Wikidata project (2012) is normalising the data in Wikipedia so that projects like DBpedia have an easier time with the raw information (no need to write alternative parsers for dates, weights, measures and simple facts!). As a result we've gone from human-readable information to machine-readable semantic-web-like information which is accessible via Linked Open Data.

Maybe the driver for semantic web data is humans trying to programmatically consume human-readable information, rather than the other way around?

I've just finished writing "High Performance Python" for O'Reilly (due August), we have a chapter on Lessons from the Field and one chap talks about his successful many-machine roll out of a complex production system using PyPy for a 2* overall speed gain. We also cover Numba, Cython, profiling, numpy etc - all the topics you'd expect.

Martin gave this paper as a talk at our PyData London conference this weekend (thanks Martin!), videos will be linked once we have them. He shares hard-won lessons and good advice. Here's my write-up: http://ianozsvald.com/2014/02/24/pydatalondon-2014/

HackerNewsLondon will occur tonight (in about an hour) http://www.meetup.com/HNLondon/ and if you turn up and blag your way in (you're meant to buy a ticket in advance, offer cash on the door to pay for your pizza!) you'll see companies talking and often talking about jobs. Just network like crazy, most people there will know people hiring. Everyone is friendly, the organisers try to help startups. Go speak to co-organiser Steve (he's an ex recruiter), he'll know who is looking.

http://www.techcityjobs.co.uk/ has a lot of London-focused tech jobs, lots of Ruby etc. Ruby is definitely in demand in London. Avoid being a waiter if you can, your skills will rust.

http://siliconmilkroundabout.com/ occurred last weekend, over 100 startups were there pitching for jobs, go through the list of companies and cherrypick, then drop them a line? The CTOs I spoke with noted the lack of Rails folk. Most of the companies had 2-8 job openings each.

Bon chance, i.

I'm working on an O'Reilly book using IPython Notebook, I'm basing my workflow on Olivier Grisel's script (with a couple of tiny fixes): https://github.com/ianozsvald/ipynbhelper which extracts the Notebook's 'markdown' blocks (which contain asciidoc which obviously won't render in the browser) & code blocks, these get exported as asciidoc for O'Reilly.

It is clunky but I'm hoping we'll get better control as nbconvert evolves, so we're experimenting with this approach.

Most of the code examples are not 'live', they're pasted in along with analysis results (the book is about high performance and parallel computing: http://shop.oreilly.com/product/0636920028963.do ), as lots of the examples are best run from a fresh VM.

Not entirely true - my Dell E6420 is certified and yet had complications (poor Suspend support, touchpad had only basic support [now fixed 2 years later]). The NVIDIA GPU was supported as was sound/internet etc. I used the official Dell Ubuntu drivers and they didn't solve the original problems (much hacking did...).

I used to travel up from Brighton (1 hour on the train, about 1.5 hours each way all in), now I live in London (and work a 5 min walk from the event).

I taught High Performance Python covering the tools you mention at PyCon 2012 (and EuroPython last year), maybe my videos+write-up will be helpful. I also cover profiling, shedskin, pyCUDA etc:

http://ianozsvald.com/2012/03/18/high-performance-python-1-f...

I'll echo Simon - just submit a short proposal (I've posted 3 :-). New speakers are encouraged as they help the community to grow. Consider adding a Poster Proposal as a backup so you could still talk about your subject even if you don't get a speaking slot. The Poster sessions last year were huge and very good.

As dagw mentions - treat it as a feature. I taught High Performance Python at PyCon, I've published all the src and a 55 page PDF covering a range of techniques to optimise a Mandelbrot example (including cython, shedskin, numpy, multiprocessing, pycuda): http://ianozsvald.com/2012/03/18/high-performance-python-1-f... With some languages this is painful, with Python you can easily profile (cProfile, line_profiler) and then work on easy or complex optimisations (both within Python and using external tools) - great flexibility for when you really need it.

I did some AI consulting work with Forward last year along with Evgeny - lovely crowd, fast growing company, lots of projects. They've got a super-nice work environment too. Worth contacting if you're around North London.

I'm also finishing StartupChile and 24pfilms is flat-sharing with us in Buenos Aires right now. We're returning to London for a while but Berlin is still on our list (nowhere else in Europe is on our list).

Quick update today (Friday 29th) - I just spotted freestanding 'check in' finger print readers in the CMI coworking building.

My guess is that registered folk get to walk through and everyone else has to explain their presence. It should be a good deterrent. Photo: https://twitter.com/ianozsvald/status/218833445760139264

Horacio (exec director) today announced the changes that I'd hinted at yesterday (I didn't know this was happening when I posted yesterday).

Changes being rolled in: * Mentorship - try to provide mentors for all * Traction Groups - get companies to start and join private self-mentorship groups (based around the prototype I organised in this round: http://ianozsvald.com/2012/05/16/mentorship-groups-in-startu... ) * Roadshow in Silicon Valley - after Demo Days take companies up to the Bay Area and run an investment/demo road show * Academy - online & face to face courses for StartupChile members to learn about business basics (e.g. lean processes, pitching, sales etc) * Top 20 Accelerator - cherry pick a top set of companies and super-accelerate them (private mentorship, focus on growth etc)

I don't know the details of the above changes, the ideas will evolve and a lot of questions are raised. What interests me is how a lot of the current programme criticisms (e.g. lack of mentorship, DemoDay not being too cool, lack of peer mentorship) should be addressed (and then some).

As stated yesterday I'm not employed by StartupChile, I'm nearly finished with Round 2. As much as I've criticised the programme in the past I continue to be impressed at how quickly they're iterating and improving.

Just to keep things connected a week later (June 27th+) a critique with pros/cons of StartupChile was posted here: http://news.ycombinator.com/item?id=4171252

I spent today (Thursday) talking with the Executive Director (Horacio) and a few of the senior staff members. I presented this morning on Lean Startup processes for the newly-arrived.

I'm in Round 2, I've been giving feedback for months to the staff. The feedback I got today about changes they're making during Round 4 (for those just arriving now) is pretty impressive. If they carry it all off then the programme will be improved by a fair margin for subsequent Rounds.

The improvements are still under discussion but ought to address many of the mentorship/fund raising/expectation issues that have been raised (at least - I have cautious optimism that that's the case).

Disclaimer - I don't work for SUP, I'm almost finished here in Round 2, I've had my complaints during the programme but generally they're reacting/improving to the significant issues. Things are improving for every new Round.