Data Science & Machine Learning Data Manipulation with NumPy and Pandas Flashcards

24 sample cards from the Data Manipulation with NumPy and Pandas deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. What does the 'method' argument in fillna (ffill / bfill) do?

   ffill (forward fill) propagates the last valid value forward; bfill (backward fill) propagates the next valid value backward.

2. In dropna(), what do axis=0 vs axis=1 and how='any' vs how='all' mean?

   axis=0 drops rows, axis=1 drops columns; how='any' drops if any value is NaN, how='all' drops only if all values are NaN.

3. What is the split-apply-combine pattern in pandas groupby?

   Split data into groups by key, apply a function (aggregate/transform/filter) to each group, then combine the results into a new structure.

4. What does df.groupby('col').agg() allow that a single aggregation does not?

   It allows applying multiple and/or different aggregation functions to one or several columns at once (e.g. {'sales':'sum','price':'mean'}).

5. What is the difference between groupby transform and aggregate?

   aggregate returns one value per group (reduced shape); transform returns a result the same shape as the input, broadcasting the group result back to each row.

6. What is a pandas pivot_table used for?

   To reshape and summarize data by grouping on index and column keys and applying an aggregation function (default mean) to the values.

7. What are the four how options of pd.merge and what does each keep?

   'inner' keeps only matching keys; 'left' keeps all left rows; 'right' keeps all right rows; 'outer' keeps all keys from both, filling missing with NaN.

8. What is the difference between pd.merge and pd.concat?

   merge combines DataFrames on key columns/indexes (database-style join); concat stacks DataFrames along an axis (rows or columns) by alignment of the other axis.

9. What do the on, left_on, and right_on parameters of merge do?

   on specifies a shared key column name; left_on/right_on specify differently named key columns in the left and right DataFrames to join on.

10. What is the difference between pivot and melt in pandas?

    pivot reshapes long to wide (spreads unique values into columns); melt reshapes wide to long (unpivots columns into key-value rows).

11. What do stack() and unstack() do to a DataFrame?

    stack() moves column labels into a (inner) row index level (wide to long); unstack() moves an index level into columns (long to wide).

12. What is a MultiIndex (hierarchical index) in pandas?

    An index with multiple levels, allowing data to be indexed and grouped by combinations of keys across more than one dimension on an axis.

13. How do you convert a column of date strings to datetime in pandas?

    Use pd.to_datetime(df['col']), which parses strings into datetime64 values enabling date operations.

14. What does the .dt accessor provide on a datetime Series?

    Access to datetime components and methods, e.g. .dt.year, .dt.month, .dt.day, .dt.dayofweek, .dt.hour.

15. What does resample() do on a time-indexed DataFrame, and how does it differ from groupby?

    resample() groups time-series data into regular time buckets (e.g. 'M', 'D') for downsampling/upsampling with aggregation; it is groupby specialized for time frequencies.

16. What is the difference between resampling downsampling and upsampling?

    Downsampling reduces frequency by aggregating into larger bins (e.g. daily to monthly); upsampling increases frequency, creating finer bins that often need fill/interpolation.

17. How do you detect and remove duplicate rows in pandas?

    df.duplicated() returns a boolean mask of duplicate rows; df.drop_duplicates() removes them (keep='first'/'last'/False controls which to retain).

18. What is the IQR method for detecting outliers?

    Compute IQR = Q3 - Q1; values below Q1 - 1.5*IQR or above Q3 + 1.5*IQR are flagged as outliers.

19. What does the z-score method for outliers measure, and what threshold is common?

    z = (x - mean) / std measures how many standard deviations a value is from the mean; |z| > 3 is a common outlier threshold.

20. How do you change a column's data type in pandas?

    Use df['col'].astype(dtype), e.g. astype('int'), astype('float'), or astype('category').

21. What does the pandas map() method on a Series do?

    It applies a function or maps values via a dict/Series element-wise to a Series, returning a new Series (useful for value substitution).

22. What is the difference between Series.apply, Series.map, and DataFrame.applymap?

    Series.map maps/substitutes element-wise (dict or function); Series.apply applies a function element-wise; DataFrame.applymap applies a function element-wise across all DataFrame cells.

23. What does the .str accessor enable on a string Series, with two examples?

    Vectorized string methods, e.g. df['name'].str.lower() and df['name'].str.strip() (also .contains, .replace, .split).

24. What is a pandas categorical dtype and why use it?

    A dtype for columns with a limited set of repeated string/label values; it stores data as integer codes mapped to categories, saving memory and speeding up grouping/comparison.

What this deck covers

Data Manipulation with NumPy and Pandas flashcards FAQ