Python Object Types - Dictionaries and Tuples

bogotobogo.com site search:

Dictionaries

Dictionaries are not sequences at all. They are mappings. Mappings are collections of objects but they store objects by key instead of by relative position.

It is best to think of a dictionary as an unordered set of key: value pairs, with the requirement that the keys are unique (within one dictionary) and must be of an immutable types, such as a Python string, a number, or a tuple. The value can be of any type including collection of types hence it is possible to create nested data structures.

When we add a key to a dictionary, we must also add a value for that key. Python dictionaries are optimized for retrieving the value when we know the key, but not the other way around. They are very similar to C++'s unordered maps. Dictionaries provide very fast key lookup.

So, mappings don't maintain any reliable left-to-right order. They simply map keys to their associated values. Dictionaries are the only mapping type in Python's core objects set. They are mutable, so they may changed in-place and can grow and shrink on demand.

Mapping

Dictionaries are coded in curly braces when written as literals. They consist of a series of key: value pairs. Dictionaries are useful when we need to associate a set of values with keys such as to describe the properties of something. For example:

>>> D = {'Composer': 'Brahms', 'period': 'Romantic', 'Symphony': 1}

We can index this dictionary by key to fetch and change the keys' associated values. The dictionary index operation uses the same syntax as that used for sequences, but the item in the square brackets is a key not a relative position:

>>> # Fetch value of key 'Composer'
>>> D['Composer']
'Brahms'
>>> 
>>> # Add 3 to 'Symphony' value
>>> D['Symphony'] += 3
>>> D
{'period': 'Romantic', 'Symphony': 4, 'Composer': 'Brahms'}
>>>

We can build the dictionary in different way starting with an empty dictionary and fills it out one key at a time.

>>> 
>>> D = {}
>>> D['Composer'] = 'Beethoven'
>>> D['Period'] = 'Classic'
>>> D['Symphony'] = 9
>>> 
>>> D
{'Symphony': 9, 'Period': 'Classic', 'Composer': 'Beethoven'}
>>> 
>>> print (D['Composer'])
Beethoven

Nesting

Let's make the previous example more complicated. We need a first name besides a last name and we want to store several pieces not just the symphony:

>>> 
>>> D = {'Composer': {'first': 'Johannes', 'last' : 'Brahms'},
         'Period': 'Romantic',
         'Piece' : ['Piano Concerto No. 1', 'Piano Concerto No. 2',
		    'Symphony No. 1', 'Symphony No. 2',
		    'Violin Concerto in D Major',
		    'Hungarian Dances'] }
>>> D
{'Piece': ['Piano Concerto No. 1', 'Piano Concerto No. 2', 
'Symphony No. 1', 'Symphony No. 2', 'Violin Concerto in D Major', 
'Hungarian Dances'], 'Period': 'Romantic', 'Composer': {'last': 
'Brahms', 'first': 'Johannes'}}
>>>

Here we have a three-key dictionary at the top: 'Composer', 'Period', and 'Piece'. But the values have become more complicated: a nested dictionary for the name to support multiple parts, and a nested for the piece to list multiple pieces. We can access the components of this structure:

>>> 
>>> # 'Composer' is a nested dictionary
>>> D['Composer']
{'last': 'Brahms', 'first': 'Johannes'}
>>> 
>>> # Index the nested dictionary
>>> D['Composer']['last']
'Brahms'
>>> 
>>> # Piece is a nested list
>>> D['Piece']
['Piano Concerto No. 1', 'Piano Concerto No. 2', 'Symphony No. 1',
 'Symphony No. 2', 'Violin Concerto in D Major', 'Hungarian Dances']
>>> 
>>> # Index the nested list
>>> D['Piece'][-1]
'Hungarian Dances'
>>> 
>>> # Expand the Piece list
>>> D['Piece'].append('Variations on a Theme by Paganini')
>>> D
{'Piece': ['Piano Concerto No. 1', 'Piano Concerto No. 2', 
'Symphony No. 1', 'Symphony No. 2', 'Violin Concerto in D Major',
 'Hungarian Dances', 'Variations on a Theme by Paganini'], 'Period': 
'Romantic', 'Composer': {'last': 'Brahms', 'first': 'Johannes'}}
>>>

Notice how the last operation here expands the nested piece list. Because the piece list is a separate block of memory from the dictionary that contains it, it can grow and shrink freely.

Let's do some clean up. In Python, when we lose the last reference to object by assigning its variable to something else. For example, all of the memory space occupied by that object's structure is automatically cleaned up for us:

>>> 
>>> D = 0
>>> D
0
>>>

Python has garbage collection feature. So, the space is reclaimed immediately as soon as the last reference to an object is removed.

Creating Nested dictionary

We want to construct the following dictionary:

{
 'a':
     {
        '1': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '2': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '3': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '4': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '5': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]}
     },
 'c':
     {
        '1': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '2': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '3': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '4': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '5': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]}
     },
 'b':
     {
        '1': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '2': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '3': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '4': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]},
        '5': {'count': 0, 'results': [{'grade': 'ok', 'rc': 0}, {'grade': 'warning', 'rc': 1}]}
    }
}

The code looks like this:

L1 = ['a','b','c']
L2 = ['1','2','3','4','5']

res_list = [
    {'grade': 'ok', 'rc': 0},
    {'grade': 'warning', 'rc': 1}
]

sub_dict = {
    'count': 0,
    'results': res_list
}

my_dict = {}
for l1 in L1:
    my_dict[l1] = {}

for l1 in L1:
    for l2 in L2:
        my_dict[l1][l2] = sub_dict

print my_dict

Sorting Keys using for Loops

Dictionary only support accessing items by key. They also support type-specific operation with method calls. Since dictionaries are not sequences, they don't maintain any dependable left-to-right order. As we've seen in the previous example, if we print the dictionary back, its keys may come back in a different order than how we typed them.

>>> 
>>> D = {'a': 97, 'b': 98, 'c': 99}
>>> D
{'a': 97, 'c': 99, 'b': 98}
>>>

What should we do if we want to impose an ordering on a dictionary's items?. One common solution is to grab a list of keys with the dictionary keys method, and sort that with the list sort method. Then, step through the result using for loop:

>>> 
>>> # Unordered keys list
>>> sortKeys = list(D.keys())
>>> sortKeys
['a', 'c', 'b']
>>> # Sorted keys list
>>> sortKeys.sort()
>>> sortKeys
['a', 'b', 'c']
>>> 
>>> # Iterate through sorted keys
>>> for k in sortKeys:
	print (k, '=>', D[k])

a => 97
b => 98
c => 99
>>>

This is a three-step process, though. We can do it with built-in sorted() in one step:

>>> 
>>> D
{'a': 97, 'c': 99, 'b': 98}
>>> for k in sorted(D):
	print (k, '=>', D[k])

	
a => 97
b => 98
c => 99
>>>

The for loop is a simple and efficient way to step through all the items in a sequence and run a block of code for each item in turn.

There is another loop, while:

>>> for c in 'blah!':
	print(c.upper())

	
B
L
A
H
!

The while loop is a more general sort of looping tool, not limited to stepping across sequences:

>>> 
>>> x = 5
>>> while x > 0:
	print('blah!' * x)
	x -= 1

	
blah!blah!blah!blah!blah!
blah!blah!blah!blah!
blah!blah!blah!
blah!blah!
blah!
>>>

Iteration and Optimization

The for loop looks like the list comprehension. Both will work on any object that follows the iteration protocol. The iteration protocol mean:

A physically stored sequence in memory
An object that generated on item at a time in the context of an iteration operation.
An object falls into this category if it responds to the iter built-in with an object that advances in response to next.
The generator comprehension expression is such an object.

Actually, every Python tool that scans an object from left to right uses the iteration protocol. And this is why sorted call used in the previous section works on dictionary directly:

for k in sorted(D):
	print (k, '=>', D[k])

We don't have to call the keys method to get a sequence because dictionaries are iterable object, with a next that returns successive keys.

This also means that any list comprehension expression which computes the cubics like the following example:

>>> cubics = [ x ** 3 for x in [1, 2, 3, 4, 5]]
>>> cubics
[1, 8, 27, 64, 125]
>>>

can always be coded as an equivalent for loop that builds the result list manually by appending as it goes:

>>> for x in [1, 2, 3, 4, 5]:
	cubics.append(x **3)

	
>>> cubics
[1, 8, 27, 64, 125]
>>>

The list comprehension, though, and related functional programming tools like map and filter, will generally run faster than a for loop.

If we ever need to tweak code for performance, Python includes tools to help us out such as time and timeit modules and the profile module.

if Tests for Missing Keys

Although we can assign to a new key to expand a dictionary, fetch a key that does not exist is still a mistake:

>>> 
>>> D = {'A': 65, 'B': 66, 'C': 67}
>>> D
{'A': 65, 'C': 67, 'B': 66}
>>> # Assigning new keys
>>> D['E'] = 69
>>> D
{'A': 65, 'C': 67, 'B': 66, 'E': 69}
>>> D['F']
Traceback (most recent call last):
  File "<pyshell#40>", line 1, in <module>
    D['F']
KeyError: 'F'
>>> 
</module></pyshell#40>

The dictionary in membership expression allows us to query the existence of a key and branch on the result with a if statement.

>>> 
>>> 'F' in D
False
>>> 
>>> if not 'F' in D:
	print('missing')

	
missing
>>>

There are other ways to create dictionaries and avoid accessing nonexistent keys. The get method and the try statement, and if/else expression.

>>> 
>>> D
{'A': 65, 'C': 67, 'B': 66, 'E': 69}
>>> 
>>> value = D.get('x', 0)
>>> value
0
>>> 
>>> value = D['x'] if 'x' in D else 0
>>> value
0
>>>

Tuples

The tuple object ( reads like toople or tuhple) is immutable list. It is an ordered sequence of zero or more object references. Tuples are coded in parentheses instead of square brackets and they support arbitrary types, arbitrary nesting, and the usual sequence operations such as len() as well as the same slicing syntax.

A tuple is much like a list except that it is immutable (unchangeable) once created. Since tuples are immutable, we cannot replace or delete any of items. So, if we want to modify an ordered sequence, we need to use a list instead of a tuple, or if we already have a tuple but want to modify it, all we need is to just convert it to a list, and then apply the changes we want to make.

>>> empty = ()
>>> type(empty)
<class 'tuple'>
>>> t = ("1")
>>> type(t)
<class 'str'>
>>> t = ("1",)
>>> type(t)
<class 'tuple'>

Above example shows how to construct tuples. Note that to create a one item tuple, we should use a comma to distinguish it from an expression with ().

>>> 
>>> # 4-tiem tuple
>>> T = (1, 2, 3, 4)
>>> len(T)
4
>>> T + (5, 6)
(1, 2, 3, 4, 5, 6)
>>> 
>>> # Indexing, slicing etc.
>>> T
(1, 2, 3, 4)
>>> T[1:]
(2, 3, 4)
>>> T[0:]
(1, 2, 3, 4)
>>> T[:-1]
(1, 2, 3)
>>>

Tuples have two type-specific callable methods:

>>> T = ('a','b','c','d','e','f','b','c','b')
>>> # Tuple methods: 'f' appears at offset 5
>>> T.index('f')
5
>>> # 'b' appears three times
>>> T.count('b')
3

The primary distinction for tuples is that they cannot be changed once created. That is, they are immutable sequences:

>>> # Tuples are immutable
>>> T[0] = 'A'
Traceback (most recent call last):
  File "<pyshell#85>", line 1, in <module>
    T[0] = 'A'
TypeError: 'tuple' object does not support item assignment
>>> 
</module></pyshell#85>

Like lists and dictionaries, tuples support mixed types and nesting, but they don't grow and shrink because they are immutable:

>>> 
>>> T =(3, 'mixed', ['A', 3, 2])
>>> T[1]
'mixed'
>>> T[2][0]
'A'
>>> T.append(44)
Traceback (most recent call last):
  File "", line 1, in 
    T.append(44)
AttributeError: 'tuple' object has no attribute 'append'
>>>

So, why do we have a type that is like a list but supports fewer operations?
Immutability!
Immutability is the whole point!
If we pass a collection of objects around our program as a list, it can be changed anywhere. But if we use a tuple, it cannot be changed. In other words, tuples provide a sort of integrity constraint that is convenient in programs of large size.

Tuples - Packing and Unpacking

Tuple packing is:

>>> T = 1, 'a', [{'NAME':'Serah', 'Age':25}]
>>> T
(1, 'a', [{'Age': 25, 'NAME': 'Serah'}])
>>>

Tuple unpacking for assignment is:

>>> x,y,z = T
>>> x,y,z
(1, 'a', [{'Age': 25, 'NAME': 'Serah'}])
>>>

Tuple unpacking requires that the list of variables on the left has the same number of elements as the length of the tuple.

Python tutorial

Ph.D. / Golden Gate Ave, San Francisco / Seoul National Univ / Carnegie Mellon / UC Berkeley / DevOps / Deep Learning / Visualization

My YouTube channel

Sponsor Open Source development activities and free contents for everyone.

Thank you.

- K Hong

Python tutorial

Sponsor Open Source development activities and free contents for everyone.

Thank you.

- K Hong

OpenCV 3 image and video processing with Python

OpenCV 3 with Python

Image - OpenCV BGR : Matplotlib RGB

Basic image operations - pixel access

iPython - Signal Processing with NumPy

Signal Processing with NumPy I - FFT and DFT for sine, square waves, unitpulse, and random signal

Signal Processing with NumPy II - Image Fourier Transform : FFT & DFT

Inverse Fourier Transform of an Image with low pass filter: cv2.idft()

Image Histogram

Video Capture and Switching colorspaces - RGB / HSV

Adaptive Thresholding - Otsu's clustering-based image thresholding

Edge Detection - Sobel and Laplacian Kernels

Canny Edge Detection

Hough Transform - Circles

Watershed Algorithm : Marker-based Segmentation I

Watershed Algorithm : Marker-based Segmentation II

Image noise reduction : Non-local Means denoising algorithm

Image object detection : Face detection using Haar Cascade Classifiers

Image segmentation - Foreground extraction Grabcut algorithm based on graph cuts

Image Reconstruction - Inpainting (Interpolation) - Fast Marching Methods

Video : Mean shift object tracking

Machine Learning : Clustering - K-Means clustering I

Machine Learning : Clustering - K-Means clustering II

Machine Learning : Classification - k-nearest neighbors (k-NN) algorithm

Machine Learning with scikit-learn

scikit-learn installation

scikit-learn : Features and feature extraction - iris dataset

scikit-learn : Machine Learning Quick Preview

scikit-learn : Data Preprocessing I - Missing / Categorical data

scikit-learn : Data Preprocessing II - Partitioning a dataset / Feature scaling / Feature Selection / Regularization

scikit-learn : Data Preprocessing III - Dimensionality reduction vis Sequential feature selection / Assessing feature importance via random forests

Data Compression via Dimensionality Reduction I - Principal component analysis (PCA)

scikit-learn : Data Compression via Dimensionality Reduction II - Linear Discriminant Analysis (LDA)

scikit-learn : Data Compression via Dimensionality Reduction III - Nonlinear mappings via kernel principal component (KPCA) analysis

scikit-learn : Logistic Regression, Overfitting & regularization

scikit-learn : Supervised Learning & Unsupervised Learning - e.g. Unsupervised PCA dimensionality reduction with iris dataset

scikit-learn : Unsupervised_Learning - KMeans clustering with iris dataset

scikit-learn : Linearly Separable Data - Linear Model & (Gaussian) radial basis function kernel (RBF kernel)

scikit-learn : Decision Tree Learning I - Entropy, Gini, and Information Gain

scikit-learn : Decision Tree Learning II - Constructing the Decision Tree

scikit-learn : Random Decision Forests Classification

scikit-learn : Support Vector Machines (SVM)

scikit-learn : Support Vector Machines (SVM) II

Flask with Embedded Machine Learning I : Serializing with pickle and DB setup

Flask with Embedded Machine Learning II : Basic Flask App

Flask with Embedded Machine Learning III : Embedding Classifier

Flask with Embedded Machine Learning IV : Deploy

Flask with Embedded Machine Learning V : Updating the classifier

scikit-learn : Sample of a spam comment filter using SVM - classifying a good one or a bad one

Machine learning algorithms and concepts

Batch gradient descent algorithm

Single Layer Neural Network - Perceptron model on the Iris dataset using Heaviside step activation function

Batch gradient descent versus stochastic gradient descent

Single Layer Neural Network - Adaptive Linear Neuron using linear (identity) activation function with batch gradient descent method

Single Layer Neural Network : Adaptive Linear Neuron using linear (identity) activation function with stochastic gradient descent (SGD)

Logistic Regression

VC (Vapnik-Chervonenkis) Dimension and Shatter

Bias-variance tradeoff

Maximum Likelihood Estimation (MLE)

Neural Networks with backpropagation for XOR using one hidden layer

minHash

tf-idf weight

Natural Language Processing (NLP): Sentiment Analysis I (IMDb & bag-of-words)

Natural Language Processing (NLP): Sentiment Analysis II (tokenization, stemming, and stop words)

Natural Language Processing (NLP): Sentiment Analysis III (training & cross validation)

Natural Language Processing (NLP): Sentiment Analysis IV (out-of-core)

Locality-Sensitive Hashing (LSH) using Cosine Distance (Cosine Similarity)

Artificial Neural Networks (ANN)

[Note] Sources are available at Github - Jupyter notebook files

1. Introduction

2. Forward Propagation

3. Gradient Descent

4. Backpropagation of Errors

5. Checking gradient

6. Training via BFGS

7. Overfitting & Regularization

8. Deep Learning I : Image Recognition (Image uploading)

9. Deep Learning II : Image Recognition (Image classification)

10 - Deep Learning III : Deep Learning III : Theano, TensorFlow, and Keras