Project Python – Free Interactive Book That Introduces Python Programming and CS
projectpython.net
projectpython.net
But there is a great, great need for REALLY well done books on intermediate python and advanced python.
- Python concurrency: multiprocessing, multithreading and asynchronous programming
- data science and numerical computation in Python
- embedded Python
- Python GUI development
- packaging and deployment of Python applications and librariesTalking about it, can you recommend a resource for that? Related: https://xkcd.com/1987/
Other than that, I don't know of a comprehensive source, in fact have been thinking about writing something myself (if time would allow...). This topic is complex because there is a lot of nuances, and there is no-one-size-fits all solution for numerous Python's use cases.
It can be complemented by:
- The Python 3 Standard Library by Example
- Python CookbookIts going to take about 3 years to write , but youll eventually be intermediate. Getting advanced is more about problem space then the actual programing language. We all piggy back on things like Hadoop eventually.
There is not automating becoming a good programmer. Its really a craft. Especially when doing it for profit.
All these intro to X things seem to gloss over the high level stuff, but I've hit a few walls in certain concepts that I just can't possibly Google. Maybe HN could help me out:
take a pandas dataframe. There's a .describe() method that I can chain to get the mean of the columns in the data frame grouped by var1:
df.groupby(df.var1).describe().mean
But then, there's also:
df.describe(df.var1).mean()
They largely do the same thing, but one gives me the mean across all variables, and the other one doesn't.
Now, how do I possibly Google this question? One method has parentheses, the other doesn't.... but It's clear I'm missing something fundamental here that I thought an intro online resource should address... but none do.
Use type() to see what you are getting:
>>> type(df.describe().mean())
<class 'pandas.core.series.Series'>
>>> type(df.describe().mean)
<class 'method'>
Something without parenthesis cannot be a method call, at least not from a normal viewpoint. Instead you are trying to access the attribute "mean" of the result of describe() if you run df.groupby(df.var1).describe().mean. This attribute might very well be a method itself (a method object).Last in that chain was describe(), so let's look at that: https://pandas.pydata.org/pandas-docs/stable/generated/panda...
It returns a "summary: Series/DataFrame of summary statistics", so calling .mean() on that, will call https://pandas.pydata.org/pandas-docs/stable/generated/panda...
Accessing .mean will instead return that method object itself.
By default pandas will return a string describing the data itself when printing a method:
>>> df.describe()
numeric n2
count 3.0 3.0
mean 2.0 3.0
std 1.0 1.0
min 1.0 2.0
25% 1.5 2.5
50% 2.0 3.0
75% 2.5 3.5
max 3.0 4.0
>>> df.describe().mean
<bound method DataFrame.mean of numeric n2
count 3.0 3.0
mean 2.0 3.0
std 1.0 1.0
min 1.0 2.0
25% 1.5 2.5
50% 2.0 3.0
75% 2.5 3.5
max 3.0 4.0>
>>> df.describe().mean()
numeric 2.00
n2 2.75
dtype: float64
I assume this is what threw you off? You are not using the "results" of .mean in further code but were just looking at them, right?Also, I noticed something very very dangerous you did:
.describe() will return a DataFrame with all those statistics. One column per numeric column of the input. One row per statistical value (like mean, std, etc). If you call .mean() on THAT DataFrame, you will get the mean of the statistical values per column, so e.g. the mean of columnA's count, mean, std, etc values. Definitely not what you want!
Instead, use either .mean() directly on your "data DataFrame" (not the one returned by .describe()). Or access the specific value from the result via .describe()['columnname']['mean'].
1. Anything that doesn’t have parens is either an attribute or a property. A property is basically an attribute, except that a computation of some sort is usually executed to set/get its value, whereas an attribute has its value set explicitly. Look at the source to identify if the result of the first code block sets an attribute named `mean` or if it has an `@property` declaration named `mean`. Either way, they’re roughly the same when using them in code.
2. The fact that both `describe()` method examples can have another function and attribute/property chained to their result means `describe()` is returning an object of some type that is not a stdlib data type.
3. The `df.groupby().describe()` is likely returning a different type of object than the second `df.describe()`, as evidenced by the first result having a `mean` attr/property, while the second has a `mean()` function.
From here, I’d start by inspecting the object type returned by `df.groupby()`, `df.groupby().describe()`, and the `df.describe()` to discover the differences.
Python objects can have both callable methods (e.g. mean()) and properties/attributes that don't need an explicit call (e.g. mean). For whatever reason, it looks like Pandas' describe() method returns a result with a different interface when called with an argument than without one.