Handle Nested YAML Structures in Python

In this tutorial, you’ll learn how to manage nested YAML structures in Python applications.

We’ll cover essential topics such as reading YAML files, converting YAML to Python objects, accessing nested values using different notations, traversing deep structures with recursive functions, and handling nested lists.

 

 

Parse Nested YAML

Read nested YAML files

To start, you’ll need to read YAML files into your Python application using the PyYAML library.

Assume you have the following YAML file:

database:
  host: localhost
  port: 5432
  users:
    - name: Ahmed
      role: admin
    - name: Fatima
      role: user
server:
  host: 0.0.0.0
  port: 8080

Code:

import yaml
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
print(config)

Output:

{'database': {'host': 'localhost', 'port': 5432, 'users': [{'name': 'Ahmed', 'role': 'admin'}, {'name': 'Fatima', 'role': 'user'}]}, 'server': {'host': '0.0.0.0', 'port': 8080}}

The code reads the config.yaml file and loads its content into a Python dictionary.

Convert YAML to Python objects

You can convert YAML data into Python objects for easier manipulation within your application.

import yaml
from collections import namedtuple
def yaml_to_object(data):
    if isinstance(data, dict):
        return namedtuple('YAMLObject', data.keys())(**{k: yaml_to_object(v) for k, v in data.items()})
    elif isinstance(data, list):
        return [yaml_to_object(item) for item in data]
    else:
        return data
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
config_obj = yaml_to_object(config)
print(config_obj)

Output:

YAMLObject(database=YAMLObject(host='localhost', port=5432, users=[YAMLObject(name='Ahmed', role='admin'), YAMLObject(name='Fatima', role='user')]), server=YAMLObject(host='0.0.0.0', port=8080))

This method converts the YAML structure into nested Python objects which allows attribute-style access.

 

Access nested values using dot notation

You can use the dot notation to access nested values after traversing the structure:

import yaml
from collections import namedtuple
def yaml_to_object(data):
    if isinstance(data, dict):
        return namedtuple('YAMLObject', data.keys())(**{k: yaml_to_object(v) for k, v in data.items()})
    elif isinstance(data, list):
        return [yaml_to_object(item) for item in data]
    else:
        return data
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
config_obj = yaml_to_object(config)
print(config_obj.database.host)

Output:

localhost

Accessing config_obj.database.host retrieves the database host using dot notation.

 

Access nested values using dictionary-style notation

Alternatively, you can access nested values using the standard dictionary method:

import yaml
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
print(config['server']['port'])

Output:

8080

Here, accessing config['server']['port'] fetches the server port value using dictionary-style indexing.

 

Traverse nested YAML using Recursive functions

Recursive functions help navigate and manipulate deeply nested YAML structures.

import yaml
def print_keys(data, prefix=''):
    if isinstance(data, dict):
        for key, value in data.items():
            print(f"{prefix}{key}")
            print_keys(value, prefix + '  ')
    elif isinstance(data, list):
        for index, item in enumerate(data):
            print(f"{prefix}[{index}]")
            print_keys(item, prefix + '  ')
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
print_keys(config)

Output:

database
  host
  port
  users
    [0]
      name
      role
    [1]
      name
      role
server
  host
  port

The print_keys function recursively traverses the YAML structure and prints each key with indentation to reflect the hierarchy.

 

Modify Nested YAML Structure

Update nested values

Changing existing values within a nested YAML structure is straightforward with Python dictionaries.

import yaml
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
config['server']['port'] = 9090
with open('config.yaml', 'w') as file:
    yaml.dump(config, file)
print(config)

Output:

{'database': {'host': 'localhost', 'port': 5432, 'users': [{'name': 'Ahmed', 'role': 'admin'}, {'name': 'Omar', 'role': 'guest'}]}, 'server': {'host': '0.0.0.0', 'port': 9090}}

Add new nested elements

You can insert new elements into the nested structure using append method:

import yaml
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
config['database']['users'].append({'name': 'Omar', 'role': 'guest'})
with open('config.yaml', 'w') as file:
    yaml.dump(config, file)
print(config['database']['users'])

Output:

{'database': {'host': 'localhost', 'port': 5432, 'users': [{'name': 'Ahmed', 'role': 'admin'}, {'name': 'Fatima', 'role': 'user'}, {'name': 'Omar', 'role': 'guest'}]}, 'server': {'host': '0.0.0.0', 'port': 8080}}Adding a new user named Omar with the role of guest extends the users list within the database configuration.

Remove nested elements

Removing elements helps in maintaining and updating the configuration as needed.

import yaml
with open('config.yaml', 'r') as file:
    config = yaml.safe_load(file)
config['database']['users'] = [user for user in config['database']['users'] if user['name'] != 'Fatima']
with open('config.yaml', 'w') as file:
    yaml.dump(config, file)
print(config['database']['users'])

Output:

{'database': {'host': 'localhost', 'port': 5432, 'users': [{'name': 'Ahmed', 'role': 'admin'}, {'name': 'Omar', 'role': 'guest'}]}, 'server': {'host': '0.0.0.0', 'port': 8080}}

This code removes the user Fatima from the users list.