Strip pandas material

TODO: remove from stats files

Author: LiamPattinson

config.yaml

@@ -65,7 +65,7 @@ episodes:
 - 03-interacting-with-tests.Rmd
 - 04-unit-tests-best-practices.Rmd
 - 05-testing-exceptions.Rmd
-- 06-testing-data-structures.Rmd
+- 06-floating-point-data.Rmd
 - 07-fixtures.Rmd
 - 08-parametrization.Rmd
 - 09-testing-output-files.Rmd

episodes/06-floating-point-data.Rmd

@@ -7,14 +7,14 @@ exercises: 5
 :::::::::::::::::::::::::::::::::::::: questions 
 
 - What are the best practices when working with floating point data?
-- How do you compare objects in libraries like `pandas` and `numpy`?
+- How do you compare objects in libraries like `numpy`?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
 - Learn how to test floating point data with tolerances.
-- Learn how to compare objects in libraries like `pandas` and `numpy`.
+- Learn how to compare objects in libraries like `numpy`.
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
@@ -40,8 +40,9 @@ but it's possible that this test could erroneously fail in future for reasons
 outside our control. This lesson will teach best practices for handling this
 type of data.
 
-Libraries like `numpy` and `pandas` are commonly used to interact with large quantities
-of floating point numbers, and they provide special functions to assist with testing.
+Libraries like NumPy, SciPy, and Pandas are commonly used to interact
+with large quantities of floating point numbers. NumPy provides special
+functions to assist with testing.
 
 ### Relative and Absolute Tolerances
 
@@ -129,6 +130,8 @@ be very different!
 
 :::::::::::::::::::::::::::::::::
 
+:::::::::::::::::::::::::::::::::::::::::::::::
+
 The built-in function `math.isclose` can be used to simplify these checks:
 
 ```python
@@ -164,16 +167,18 @@ def test_estimate_pi():
 
 :::::::::::::::::::::::::::::::::
 
+:::::::::::::::::::::::::::::::::::::::::::::::
+
 ### NumPy
 
 NumPy is a common library used in research. Instead of the usual `assert a ==
 b`, NumPy has its own testing functions that are more suitable for comparing
 NumPy arrays. These functions are the ones you are most likely to use:
 
-- `numpy.testing.assert_array_equal` is used to compare two NumPy arrays or array-like objects (such as list, tuples, etc).
-- `numpy.testing.assert_allclose` is used to compare two NumPy arrays or array-like objects with a tolerance for floating point numbers.
-
-These may also be used on individual floating point numbers if you choose.
+- `numpy.testing.assert_array_equal` is used to compare two NumPy arrays for
+  equality -- best used for integer data.
+- `numpy.testing.assert_allclose` is used to compare two NumPy arrays with a
+  tolerance for floating point numbers.
 
 Here are some examples of how to use these functions:
 
@@ -206,59 +211,23 @@ def test_numpy_arrays_with_tolerance():
     np.testing.assert_allclose(array1, array2, atol=1e-3)
 ```
 
-::::::::::::::::::::::::::::::::::::: callout
-
-### Data structures with numpy arrays
-
-When you have data structures that contain numpy arrays, such as lists or dictionaries, you cannot use `==` to compare them.
-Instead, you can use `numpy.testing.assert_equal` to compare the data structures.
-
-```python
-def test_dictionaries_with_numpy_arrays():
-    """Test that dictionaries with numpy arrays are equal"""
-    # Create two dictionaries with numpy arrays
-    dict1 = {"a": np.array([1, 2, 3]), "b": np.array([4, 5, 6])}
-    dict2 = {"a": np.array([1, 2, 3]), "b": np.array([4, 5, 6])}
-    # Check that the dictionaries are equal
-    np.testing.assert_equal(dict1, dict2)
-```
-
-::::::::::::::::::::::::::::::::::::::::::::::::
+The NumPy testing functions can be used on anything NumPy considers to be 'array-like'.
+This includes lists, tuples, and even individual floating point numbers if you choose.
+They can also be used for other objects in the scientific Python ecosystem, such
+as Pandas Series/DataFrames.
 
+:::::::::::::::::::::::: callout
 
-### pandas
+The Pandas library also provides its own testing functions:
 
-Pandas is another common library used in research for storing and manipulating datasets.
-Pandas has its own testing functions that are more suitable for comparing Pandas objects.
-These two functions are the ones you are most likely to use:
-- `pandas.testing.assert_frame_equal` is used to compare two Pandas DataFrames.
-- `pandas.testing.assert_series_equal` is used to compare two Pandas Series.
+- `pandas.testing.assert_frame_equal`
+- `pandas.testing.assert_series_equal`
 
+These functions can also take `rtol` and `atol` arguments, so can fulfill the
+role of both `numpy.testing.assert_array_equal` and
+`numpy.testing.assert_allclose`.
 
-Here are some examples of how to use these functions:
-
-```python
-
-def test_pandas_dataframes():
-    """Test that pandas DataFrames are equal"""
-    # Create two pandas DataFrames
-    df1 = pd.DataFrame({"A": [1, 2, 3], "B": [4, 5, 6]})
-    df2 = pd.DataFrame({"A": [1, 2, 3], "B": [4, 5, 6]})
-    # Check that the DataFrames are equal
-    pd.testing.assert_frame_equal(df1, df2)
-
-def test_pandas_series():
-    """Test that pandas Series are equal"""
-    # Create two pandas Series
-    s1 = pd.Series([1, 2, 3])
-    s2 = pd.Series([1, 2, 3])
-    # Check that the Series are equal
-    pd.testing.assert_series_equal(s1, s2)
-```
-
-There is no equivalent to `np.assert_allclose` in Pandas. If you need to compare DataFrames
-or Series containing floating point data, it is recommended to use the `np.testing` functions directly
-on the Pandas objects.
+::::::::::::::::::::::::::::::::
 
 
 ::::::::::::::::::::::::::::::::::::: challenge 
@@ -299,79 +268,15 @@ def test_calculate_cumulative_sum():
 
 :::::::::::::::::::::::::::::::::
 
-### Checking if Pandas DataFrames are equal
-
-In `statistics/stats.py` add this function to calculate the average score of each player in a Pandas DataFrame:
-
-```python
-import pandas as pd
-
-def calculate_player_average_scores(df: pd.DataFrame) -> pd.DataFrame:
-    """Calculate the average score of each player in a pandas DataFrame.
-    
-    Example input:
-    |   | player  | score_1 | score_2 |
-    |---|---------|---------|---------|
-    | 0 | Alice   | 1       | 2       |
-    | 1 | Bob     | 3       | 4       |
-
-    Example output:
-    |   | player  | score_1 | score_2 | average_score |
-    |---|---------|---------|---------|---------------|
-    | 0 | Alice   | 1       | 2       | 1.5           |
-    | 1 | Bob     | 3       | 4       | 3.5           |
-    """
-
-    df["average_score"] = df[["score_1", "score_2"]].mean(axis=1)
-
-    return df
-```
-
-Then write a test for this function by comparing Pandas DataFrames.
-
-Hint: You can create a dataframe like this:
-
-```python
-df = pd.DataFrame({
-    "player": ["Alice", "Bob"],
-    "score_1": [1, 3],
-    "score_2": [2, 4]
-})
-```
-
-:::::::::::::::::::::::: solution
-
-```python
-import pandas as pd
-from stats import calculate_player_average_scores
-
-def test_calculate_player_average_scores():
-    """Test calculate_player_average_scores function"""
-    df = pd.DataFrame({
-        "player": ["Alice", "Bob"],
-        "score_1": [1, 3],
-        "score_2": [2, 4]
-    })
-    expected_result = pd.DataFrame({
-        "player": ["Alice", "Bob"],
-        "score_1": [1, 3],
-        "score_2": [2, 4],
-        "average_score": [1.5, 3.5]
-    })
-    pd.testing.assert_frame_equal(calculate_player_average_scores(df), expected_result)
-```
-
-:::::::::::::::::::::::::::::::::
-
-
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- When comparing floating point data, you should use relative/absolute tolerances instead of testing for equality.
-- Numpy arrays cannot be compared using the `==` operator. Instead, use `numpy.testing.assert_array_equal` and `numpy.testing.assert_allclose`.
-- Pandas DataFrames and Series should be compared using `pandas.testing.assert_frame_equal` and `pandas.testing.assert_series_equal`.
+- When comparing floating point data, you should use relative/absolute
+  tolerances instead of testing for equality.
+- Numpy arrays cannot be compared using the `==` operator. Instead, use
+  `numpy.testing.assert_array_equal` and `numpy.testing.assert_allclose`.
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 

learners/files/06-floating-point-data/data_structures.py

@@ -0,0 +1,27 @@
+import numpy as np
+
+def test_numpy_arrays():
+    """Test that numpy arrays are equal"""
+    # Create two numpy arrays
+    array1 = np.array([1, 2, 3])
+    array2 = np.array([1, 2, 3])
+    # Check that the arrays are equal
+    np.testing.assert_array_equal(array1, array2)
+
+
+def test_2d_numpy_arrays():
+    """Test that 2d numpy arrays are equal"""
+    # Create two 2d numpy arrays
+    array1 = np.array([[1, 2], [3, 4]])
+    array2 = np.array([[1, 2], [3, 4]])
+    # Check that the nested arrays are equal
+    np.testing.assert_array_equal(array1, array2)
+
+
+def test_numpy_arrays_with_tolerance():
+    """Test that numpy arrays are equal with tolerance"""
+    # Create two numpy arrays
+    array1 = np.array([1.0, 2.0, 3.0])
+    array2 = np.array([1.00009, 2.0005, 3.0001])
+    # Check that the arrays are equal with tolerance
+    np.testing.assert_allclose(array1, array2, atol=1e-3)