Ask HN: Data visualization consultant | Hacker News Reader

Ask HN: Data visualization consultant

81 pointsmxmpawn31 comments

I'm working as a software engineer (mostly web, mostly backend). I've worked some years as a freelancer (now working at a part time job + contractor job) always working on the backend side.

I'm getting bored of this kind of job and I'm exploring data visualization. In my job (news media) my boss started to ask me to make some things with d3.js and I liked it, with the plus side that the "deliverable" is something people can see and if it's done well it could be easily shared and generate visits.

So I started to think about learning more about d3.js, build and show some things with it in my free time and then start looking for clients.

Anyone here already doing something like this? what do you think about it? what are your experiences?

View on HN

I would recommend starting with Stephen Few's books: Show Me the Numbers and Information Dashboard Design: Displaying Data for At-a-Glance Monitoring to get an idea of visualization from a business needs perspective. These are easy reads.

Few is heavily influenced (and cites) Edward Tufte - Tufte is probably the definitive reference for communicating data with visual techniques. If you are interested in pursuing scientific visualization, you probably need to spend some time with Tufte. I never have properly done this myself.

These sources will give you a solid foundation without being tied to specific tech. You'll probably find lots of libraries built on top of d3.js that implement the basic ideas of both authors. I have been meaning to look at data exploration examples (e.g., stuff with visualization examples in R) and translate to different front/backend. That might be neat too.

Build some examples where the visualization provides real insight to a defined business problem and you will probably score some new opportunities for yourself.

Good luck.

Very nice feedback, thanks!. I know Tufte (here in HN there was a topic about his opinion on pie charts) but I've never read something from Stephen Few. I'll definitely check it out.

Regarding the building of some examples, yes, I've already made some of them for the news media job I have, but I'm doing a contract job for a real estate aggregator company (like trulia but from Argentina) and I'm interested in all things regarding public transport. So I think I'll explore these topics and see what sticks.

I've been learning this stuff for work some. I've found that Good Charts: The HBR Guide to making Smarter More Persuasive Data Visualizations[1] to be helpful.

Also the Wall Street Guide to Info Graphics[2] is a good 'recipe' book for graphs/charts.

1. https://www.amazon.com/Good-Charts-Smarter-Persuasive-Visual... 2. https://www.amazon.com/Street-Journal-Guide-Information-Grap...

I had previously contracted for Infinera working on their NMS suite. My contribution was a PoC running on D3 and Angular, using Semantic Topology that allows users to navigate any node or link in their network with zero clicks (scrolling and panning gets them anywhere à la Google Maps).

The hardest, most time-consuming part was prototyping because the tools out there (Framer, Principle, Quartz Composer, Adobe XD etc) are all optimized to work with lists and grids, not visualisations. I ended up creating three fully-functional prototypes in HTML and JS before finalizing on one. It was hard to get good feedback from the teams without live, interactive prototypes. There is also nothing like a standard library of well-designed components that can be grouped together to quickly mockup custom visualizations, like we have in Bootstrap for HTML frontends.

Another problem was performance; Angular, out-of-the-box, could not keep up with the amount of data coming through their pipes.

All in all, it was challenging and very, very fun to do. When it finally worked and people got their hands on it, it blew everyone away. If I were to do it all over again, I’d focus a lot more on explaining the challenges and setting realistic expectations. Custom visualisations are not 5-minute WordPress installs.

The work is under NDA but if you feel the need to discuss anything, my email is in my profile.

Some people I follow on twitter who are actively involved in DataVis:

Lisa Charlotte Rost (@lisacrost), Micah Stubbs (@micahstubbs), Jim Vallandingham (@vlandham), Nicky Case (@ncasenmare), Nadieh Bremer (@NadiehBremer)

we're putting some viz into our product now.

to your point, non-technical should be taught to understand that this is challenging data and graphics programming (which means there is a whole dev cycle associated with it), not photoshop or indesign style work where the tool is itself a program that you instruct to do things by pointing or scripted actions.

most people have no idea what the difference is.

I have been doing freelance data viz with Tableau for a little over a year now and love many aspects of the work. I am a full time revenue analyst and find clients on and off for freelance.

I started with fiver.com where I didn't really make any money but I learned many things and was able to do work for Dell and USC. It has more than paid for itself to have their logos on my website, so it might be worth a look. Some will discourage the low rate approach but it has worked for me to gain experience over dollars in the beginning and I can currently charge market average or better because of the experience.

Some other important lessons I have learned:

1. Turn down work if you are too busy. Its worse to hurt a client relationship than to pass up some work, to the point where you will lose much more in the long run.

2. Understand all the specific requirements before accepting a contract.

3. Be realistic with the timetable, don't underestimate any complexities of the data or requests.

4. Make sure the client understands the iterative process and is available for feedback along the way.

5. Reconcile the vizzed data back to the source. Tools and programming can do unexpected things to data if you don't fully understand all of the processes that produce the viz.

I freelance because I love data and helping others understand data through visualization. I also have found that data visualization is my passion when it comes to career interests. If you are discovering the same while developing with d3.js, I would recommend putting all your efforts into it. Good Luck!

Sure, there are a bunch of places and ways you can do this.

For a start, companies of all shapes and sizes have or are building internal data technology teams. Almost any large company wants to do things with data that exceed what they can achieve by buying services. So they chuck all the data in a warehouse, and then find themselves in need of a way of interpreting and visualising it. You get fully-fledged internal facing products built on that premise all the time.

Alternatively, visualising data is a huge part of what modern media companies do. Examples range from Quartz, who built Atlas[0], amongst other things, to Bloomberg, whose business facing financial intelligence services have to make large volumes of data intelligible at a glance.

Beyond just the media, if you look at products like Facebook's Ad Platform or Google's DFP or Analytics, the UIs devote a ton of effort towards visualising data. It's probably harder to find tech companies that don't do large amounts of this than companies that do.

I haven't seen it as a consultancy role, but it is certainly something you could do on a contract basis with some companies if you wanted to. As with any software contracting, if you can show a great strength in a narrow niche you will command near-monopoly pricing for your services.

[0] http://atlas.qz.com/

FYI there is indeed a strong demand, and having the ability to develop full-stacked web apps is a nice plus as well.

You can check this Google Group[0] to see actual requests (full-time jobs, part-time jobs and limited prestations).

[0] https://groups.google.com/forum/#!forum/data-vis-jobs

We're already doing something like this, although we're currently using MS Power BI and will likely move to Tableau soon.

Our job as a company is to make the lives of business leaders easier, so business intelligence is one way of doing that. We ask a series of questions of the data, build out the visuals and then present findings and make strategic recommendations.

We started off doing it for ourselves in the beginning so have been doing it for almost a year now. My business partner has been doing this work previously for a couple of years in a full time role.

Happy to do a skype to share info with you. Would be potentially open to sending you some work once we build our portfolio up a bit better too. Finding freelancers to work on this reliably and dependably is quite hard.

Great!, I didn't find your email in the profile. Mine is there, glad to have a skype.

Hey, your email isn't showing up either. You can email me at kam[at]stratagem[dot]io

I'll send you my calendar when i get your email.

Let me know if you need any help with the transition to Tableau. Been addicted for about 3 years now.

For sure. Would love to share ideas with you. Do you use anything dedicated for ETL? Also, for Tableau do you use the desktop version or server/online?

I generally steer my clients to a BI partner that does the ETL so I can focus on visualization. I use Desktop 10.0/Server 10.0 and some Online depending on the needs. I'll send you an email so you have my contact info.

If you don't mind talking about, would be interested to hear why you guys are moving from PowerBI.

Sure.

There are a number reasons. None of which individually, are huge or significant but as a combination are enough to make us consider something else.

1. Tableau has more polish as a product. The company has been around longer and has more time to refine its craft. Microsoft's product on the other hand while very impressive given how little time they've had in this area is a little clunky in comparison. Also the final visuals just seem to look better and the dashboard has better organisation which makes for a better end user experience.

2. Tableau Talent. There seem to be many more people that know about and have worked with Tableau in comparison to MS Power Bi. Much easier to outsource the work for Tableau than for the Microsoft product

3. Knowledge Base. It constantly seems to be much easier to find answers to common questions with Tableau than with Power BI. Microsoft have taken their typical Microsoft approach with things and it's quite fragmented, as are all their products.

4. Connectors. Not as a big a deal by any means but Tableau generally tends to have more connectors. Microsoft Power BI only just got an Amazon Redshift connector, whereas Tableau has had it for ages.

5. No Mac Client for Microsoft Power BI

Hope that helps.

HI Kam

I am a DWBI veteran. On the ETL side I have got excellent experience on Talend and kettle. I have expereince on data visualization, data modeling, data warehousing, BI, ETL.

please do have a look at some of my data visualizations work. Done using the community (free) version of open source BI tool [Helical Insight](http://www.helicalinsight.com)

[Travel Dashboard](http://www.helicalinsight.com/hi/?dir=Travel_Dashboard&file=...)

[Sales Dashboard](http://www.helicalinsight.com/hi/?dir=Sales_Dashboard&file=s...)

[Infograph](http://www.helicalinsight.com/hi/?dir=CV_Creator/CV_Creator_...)

One of my client implementation was on Envision GLobal. They are into training and leadership domain wherein they conduct training followed by survey on limesurvey. I had done data modeling for their reporting database, created ETL for data loading from limesurvey database into reporting database, created a GUI using which client can themselves trigger and run these ETL with configurable parameters, integration of this into Drupal paltform for end client access etc.

Reports were created for each and every individual user talking about his traits as well as comparison with other colleagues (very similar to annual report with images, text, formatted text, charts), ability to edit the report via UI itself, save and email the reports (email scheduling), exporting etc. Also proper user role management and data security was provided.

I would love to speak with you and discuss of possible working opportunity with you. You can get in touch with me at

nikhilesh(at)helicalinsight.com

I am also dropping a mail to you

Thanks NikhilEsh. I will look out for your email and send across my calendar to you when i get back into the office next week. It'd be good to chat.

A large part of my job involves creating data visualizations to summarize work to non-technical audiences.

I found Udacity's course on data visualizations to be a great resource for nailing the basic rules of effective data visualization and communication.

Something I've learned as I've worked is to have some way to quickly try ideas and some way to productionize the insights you've found for broad consumption. So I really like using pyplot or tableau for really fast insight gathering and visualization creation. Once I've got what I think are good visualizations, I can then decide what makes the most sense for production. Sometimes this means a custom D3 web solution, other times a tableau dashboard is just fine; it all depends on the project.

And separate from all this, make sure you frequently include fresh sets of eyes for your visualizations. It's easy to make something that is obvious and useful to you, but is hard to understand for others.

Good luck consulting. I don't have much help in getting that started. I am really passionate about this field so it's always great to hear about others who enjoy visualizations as well!

I've done it, usually it's people stumbling upon the visualisations I did for my own interest (from my website, Twitter, or d3's gallery) finding they'd like something close to them and contacting me directly. Maps most often.

I like it, it's like paid leisure, and the more you do it, the most likely you already have something that almost does exactly what they want and just needs a few changes.

I wouldn't know where to start looking for clients though.

I've looked at your work, you've got pretty great pieces.

If you don't mind me asking, which kind of organizations are usually interested in your work?

Thanks!

For me it's usually smallish organisations, startups, sometimes associations, who want a map that looks better than a run-of-the-mill Google maps with plain markers, really. So they see something like https://ssz.fr/voies/ or one of the variations of https://ssz.fr/territoire/ and ask for something like it that shows their data in an adequate way (and hopefully a good looking way too, using their colour scheme, etc).

After that they sometimes ask me if I'm available for more map-related projects or iterative improvements to the first maps (and I am as long as they're small enough projects which won't take me more than a week or two).

I think larger companies just don't email a random author after they find their address on a map, but that's fine because they're less interesting to work with in my opinion.

I have been doing a similar kind of job. I really recommend this group for finding contracts: https://groups.google.com/forum/#!forum/data-vis-jobs

The easy part is:

- if you have a few projects, it is easy to show that you are competent and for them to decide if it's style they want (I was luring with this one: http://p.migdal.pl/tagoverflow/ and a few other things on my GitHub page),

- people are happy as they receive something visual (appeal of a good project is obvious) and, often, cooler that they expected.

The bad part is that many projects were short - 1-2 days of work. Even if charged at a high rate, still a lot of time is spent on switching before projects, communication, etc.

...and some links for learning: http://p.migdal.pl/2016/02/09/d3js-icm-kfnrd.html

Yes! The approach I took was to build a product that solves a broad problem (in my case geographic visualisation) and showed that to get clients who wanted something similar but tweaked for their use case. Of course they then ask 'is it possible to do this other thing' and all of a sudden you're doing data vis consulting.

As an aside, I'd suggest selling the data science too if you have the skills since data viz isn't just about showing data but figuring out how to show it to answer the relevant business questions - they go hand-in-hand and it's both a more compelling sell and adds more value to deliver the full stack solution instead of just the viz piece.

Good luck!

My initial interest was in data science but it seems hard to get freelance gigs. But maybe the viz thing could help for smoothing the field.

We use d3.js for a variety of our clients. We're always on the lookout for freelancers. If you're interested in talking more about the type of work we do, feel free to email me.

Eric eric@frac.tl

please do have a look at some of my data visualizations work. Done using the community (free) version of open source BI tool [Helical Insight](http://www.helicalinsight.com)

[Travel Dashboard](http://www.helicalinsight.com/hi/?dir=Travel_Dashboard&file=...) [Sales Dashboard](http://www.helicalinsight.com/hi/?dir=Sales_Dashboard&file=s...) [Infograph](http://www.helicalinsight.com/hi/?dir=CV_Creator/CV_Creator_...)

You can get in touch with me at nikhilesh(at)helicalinsight.com

There's actually an entire company focused on exactly this (I know because I used to work there).

You should check them out: https://www.graphiq.com

Thanks, I already knew about graphiq and it's a great inspiration, my idea, if the consulting/freelancing works out, would be trying to pivot into a business model similar to graphiq.

What can you tell us about your experience working for them? Do you know how it got started?

I would also like this job.

Look at Eyeo talk videos like Ben Fry's. There's a lot of data vis that goes through there.

Reply on news.ycombinator.com