Pandas with NumPy and Matplotlib

Pandas provides a Python library such as IPython toolkit and other libraries, the environment for doing data analysis in Python.

We're not going to do a lot in this article but presents a simple example for reading in a data file and do a little bit of data manipulation using NumPy. Then, we'll draw a simple scatter plot.

We'll use Jupyter notebook throughout this material, and the notetook I used is available from Gihub : H-R-Diagram-Pandas-Matplotlib.ipynb.

The data we will reads in is about H-R diagram which looks like this:

pic. source Hertzsprung–Russell diagram.

Here is my Jupyter notebook (Gihub : H-R-Diagram-Pandas-Matplotlib.ipynb)

Let's read in the data:

Since the data has blank (space) data, we need to clean it up. In this case, I just removed the row though here are couple of ways to handle the missing data (scikit-learn : Data Preprocessing I - Missing / Categorical data)).

Now we do not have flawed data, and we're ready to plot.

Here is the scatter plot via Matplotlib:

Just for fun, in this section, we'll do classification via Scikit-learn's DBSCAN which is one of the unsupervised clustering algorithm.

The code is available from Gihub : H-R-Diagram-Pandas-Matplotlib.ipynb:

Here is the plot after clustering. But still not catching the "White Dwarf":

A dataset we're going to read in is "Breast Cancer Wisconsin" dataset.

It contains 569 samples of malignant and benign tumor cells.

The first two columns in the dataset has the unique ID numbers of the samples and the corresponding diagnosis (M=malignant, B=benign), respectively. The columns 3-32 contain 30 real-value features that have been computed from digitized images of the cell nuclei, which can be used to build a model to predict whether a tumor is benign or malignant.

Let's read in the dataset directly from the UCI website using pandas:

Then, let's assign the 30 features to a NumPy array X, and corresponding diagnosis (M=malignant, B=benign) to y.

Using LabelEncoder, we can transform the class labels from their original string representation ( B and M ) into integers (0 and 1):