The book should be in print by (hopefully) the end of this year, or definitely by Jan 2017. The content will not change significantly, but there is will be minor fixes and a lot of proof reading.
I understand there is always one more library or topic that could be included...
.. but with that acknowledged, what do you think of sqldf as an alternative to dplyr? You mention that dplyr is a bit easier (within the context of being specialized for data analysis). I'd have trouble weighting in because I don't use R all that much, but I do really like the python "equivalent" pandasql.
Also, I've used SQL for a long time, so I'd have trouble at this point really knowing what's "easier" for someone new to both, but I do often find it easier to use SQL than do data frame operations in pandas. dplyr seems to be a closer cousin to standard SQL, so the difference might not be quite as great.
SQLite seems like it has the guts to be the standard libdataframe.c for R, Python, Julia, etc. As a side benefit it already has a super consistent API (a.k.a. SQL).
Also SQL is a substantially inferior API for data analysis. (Not because it's a bad language, but again because that's not what it's designed for)
Because pandasql returns a data frame from a data frame (not sure if this is the case with R), I find it relatively easy to do data things with sql and data analysis with python. However, that's not a huge surprise since I've been using SQL for a while but don't know the pandas or R data frame syntax especially well.
I'm not sure why sqlite was chosen - could it have to do with the in-memory nature of dataframes? So far, my use of sql with data frames has been pretty generic, so I haven't bumped up any implementation specific SQL issues.
Also window functions are really useful for data analysis, and they are much easier to express in dplyr than they are in SQL (at the cost of being slightly less general).
The biggest issue I found was that sqldf was significantly slower than dplyr and other alternatives.
I started trying to mess about with something I was calling sqldf2. Didn't get very far, but there is some perhaps somewhat useful benchmarking in the R script here:
And of course, thanks for another great book, it's helpful for learning R but I'm always enlightened by how thoroughly you explain the general concepts (e.g. Relational data and joins). Have heard a few people on faculty speak enthusiastically about the book even as I hold out for more adoption of Python :)
If I'm designing a DB or even just an SQL example, I'll run the code on my local machine (psql + the Postgres app [2]) or if I'm lucky, the client already has a server running Postgres and I can run it there instead. All SchemaSpy then needs is access to the DB and voila, interactive example.
I have a bit of a nitpick about chapter 13 on "relational data", in which I believe you are consistently misusing the technical term "relation" to refer to the relationship between two data sets. In the context of relational database theory, "relation" is just another word for "table" (although it connotes more mathematical formalism).
I think it is worth respecting the precise technical usage in this case: Consider a student who might read your book and be told that "relations are always defined between a pair of tables," and that "a primary key and the corresponding foreign key in another table form a relation." The same student might also stumble across the wikipedia page for the relational model and learn that "a relation is defined as a set of tuples that have the same attributes," and that the relational model "organizes data into one or more tables (or relations)."
The relational data model - the set of relation variables - is thus the "equation that defines what is going on in your company" and what the DBMS puts in - the set of relation values, usually abbreviated to relations - can be seen as "the history of what happened at your company".
R is a terrible programming language. It's slow and syntactically inconsistent. For interactive statistical analysis it can be OK, but anything beyond a small program becomes unmanageable quickly. In particular, if you want to manipulate strings or hierarchical data structures quickly in R, good luck.
For ML, scikit-learn is almost always sufficient. For statistics, statsmodels is OK but very underdeveloped compared to what is available in R. IMO plotting is equally painful in both Python and R (seaborn is a good Python library to ease the pain if you haven't seen it).
Personally, I write everything in Python and call out to R as infrequently as possible using rpy2, usually only for specific statistical routines or bioinformatics-specific libraries.
Lot of the exercises, especially in the exploratory data analysis part are "why is blah?" or "is there a relationship in blah?"
I think I know the answers, but it would be nice to be able to check if I see what I'm supposed to se in the data.
I've been using r4ds for the past couple weeks. This is the first time I've really understood how everything in R and the tidyverse fits together. I am really enjoying the book. It has already helped me immensely.
Seriously, thank you for writing this.
I came from Python using iPython. I missed them for the first few weeks, but now I can't switch from RStudio. It really is just such a great tool for data science.
You use back ticks to make your code chunks.
For example: Some random Markdown text here is treated as a text block in Juypter
```{r}
summary(cars)
```
Then some further text goes here.
Jupyter has some conveniences, but the tradeoffs aren't worth it for me. Working in a web browser has much less power, when it comes to keyboarding, than Atom/Sublime. And I generally don't need to interact with my data; I know what I'm outputting, I just want to show the results to readers alongside my code. I don't use RStudio but RMarkdown is easy to run from the command line.
By contrast, Jupyter requires (AFAIK) working within the browser and, when you save the file, you get a huge jumble of JSON, which is how the notebook is serialized. I tend to write a lot of vignettes/explorations and the need to full-text grep them is important to me and is not feasible when the text content is saved as JSON.
You can convert the Notebooks to different formats. Though they tend to have to be fixed up a bit for a script (Which I also might need to in R to turn it into a script)