Hello and welcome back! In our last post, we explored on high level some common OpenStack components like Nova and Keystone. Today, we'll dive deeper into how OpenStack structures its authentication and authorization modules, particularly focusing on the Keystone component and its implementation using Django.
Note: code use in this post can be found here
Authentication and Authorization
Authentication systems are the gatekeepers of digital systems, ensuring that users are indeed who they claim to be. Think of logging into platforms like Facebook or X (formerly Twitter), you input your email and password, and behind the scenes, these platforms validate your credentials by comparing them with stored records. If the credentials match, you're authenticated.
Authorization, on the other hand, determines what an authenticated user is allowed to do. For example:
Can the user read posts?
Can they create new posts?
Are they allowed to manage system settings?
These two systems work together to ensure secure access and proper resource management.
Keystone: OpenStack's Identity Service
In OpenStack, Keystone is the cornerstone of authentication and authorization. It provides:
Authentication: Verifying user identity using a username and password (or other methods like tokens or OAuth).
Authorization: Determining user permissions via roles and scopes.
Token Management: Issuing, validating, and revoking tokens that users use to interact with OpenStack APIs.
Authentication in Keystone
When a user wants to access OpenStack services, they provide their credentials (username and password). Keystone validates these credentials by:
Checking against the identity backend, such as an LDAP server or SQL database.
Generating an authentication token upon successful verification.
This token is then used by the user for all subsequent API calls to avoid re-entering credentials repeatedly. The authentication flow looks like this:
User sends their username and password to Keystone.
Keystone verifies the credentials.
A token is issued to the user.
That was quite straight forward right? Yes it is!
Authorization in Keystone
Once authenticated, the token is used to authorize the user's access to specific resources. The level of access is determined by roles and scopes:
Roles: Define what actions a user can perform (e.g. admin, manager, project member).
Scopes: Limit the extent of the user's access (e.g., specific projects or domains).
For example:
An admin might have full control over all projects.
A project member might only be able to manage resources within their assigned project.
Django and Openstack Keystone
Problem Statement
We need to implement a seamless integration between Django and OpenStack's Keystone component to address the following requirements:
Automatically create a new user in OpenStack when a user is created in the Django database.
When a user successfully logs into Django, they should also be authenticated on OpenStack on project scope pattern.
Set up Django project
To interact with OpenStack's Keystone component, we will utilize Django REST Framework, Docker, and httpx for making API calls. Let's start by setting up our Django project. Follow these steps to get everything configured:
Create and Activate Virtual Environment
#create virtual env python -m venv env #activate virtual env source env/bin/activateInstall Required Dependencies
pip install django djangorestframework httpx python-dotenv uvicorn psycopg2-binary celery redis cryptographyCreate ‘requirements.txt‘ file to be used by docker
pip freeze > requirements.txtCreate Django project - I will call the project openstack
django-admin startproject openstack .Create dockerfile and docker-compose file
FROM python:3.12.0a3-slim-buster # Set environment variables ENV PYTHONUNBUFFERED 1 ENV PYTHONDONTWRITEBYTECODE 1 # Create and set working directory WORKDIR /app # Install system dependencies RUN apt-get update && apt-get install -y \ gcc \ python3-dev \ musl-dev \ libpq-dev # Install Python dependencies COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt # Copy project files into the container COPY . /app/ # Make port 8000 available to the world outside this container EXPOSE 8000# docker-compose-dev.yaml services: web: container_name: django build: . command: ["uvicorn", "openstack.asgi:application", "--host", "0.0.0.0", "--port", "8000", "--reload"] volumes: - .:/app ports: - "8000:8000" depends_on: - db env_file: - ./.env networks: - openstack-net db: container_name: postgres image: postgres:${PG_VERSION:-16} volumes: - postgres_data:/var/lib/postgresql/data environment: - POSTGRES_DB=${DB_NAME} - POSTGRES_USER=${DB_USER} - POSTGRES_PASSWORD=${DB_PASSWORD} networks: - openstack-net redis: container_name: redis image: "redis:alpine" ports: - "6379:6379" networks: - openstack-net celery: container_name: celery build: . command: ["celery", "-A", "openstack", "worker", "--loglevel=INFO"] volumes: - .:/app depends_on: - db - redis env_file: - ./.env networks: - openstack-net volumes: postgres_data: networks: openstack-net:Configure Environment Variables
Create an
.envfile to store sensitive configuration data like database credentials and OpenStack detailsDB_NAME='openstack_db' DB_USER='openstack_user' DB_PASSWORD='secure_password' OPEN_STACK_AUTH_URL='https://200.225.45.100:5000' OPEN_STACK_USER_DOMAIN_NAME='default' OPEN_STACK_PROJECT_DOMAIN_NAME='default' SECRET_KEY='*bbrag07n)njowffakrnjsk!3h5h(w$d(qpxod3wh' OPEN_STACK_ADMIN_PROJECT_ID='c94fe17bd628c19' OPEN_STACK_ADMIN_USERNAME='admin' OPEN_STACK_ADMIN_PASSWORD=''Replace the db declaration in settings.py file in openstack folder to point django to the db container
DATABASES = { "default": { "ENGINE": "django.db.backends.postgresql", "NAME": os.getenv('DB_NAME'), "USER": os.getenv('DB_USER'), "PASSWORD": os.getenv('DB_PASSWORD'), "HOST": 'db', "PORT": "5432", } }Now let’s configure celery to handle background tasks
In the Django project's root directory - openstack folder, create a file named celery.py and add the content below
from __future__ import absolute_import, unicode_literals import os from celery import Celery # Set the default Django settings module for the 'celery' program. os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'openstack.settings') app = Celery('openstack') # Using a string here means the worker doesn't have to serialize # the configuration object to child processes. # Namespace 'CELERY' indicates all celery-related configs should be prefixed with 'CELERY_'. app.config_from_object('django.conf:settings', namespace='CELERY') # Auto-discover tasks in installed apps. app.autodiscover_tasks()Update the "init.py" file in the Django project's root directory to include Celery
from __future__ import absolute_import, unicode_literals # This will make sure the app is always imported when # Django starts. from .celery import app as celery_app __all__ = ('celery_app',)Add the following Celery settings in the settings.py in openstack folder
CELERY_BROKER_URL = 'redis://redis:6379/0'
Build and Run the Docker Containers: To bring up the services, use the following commands
# Build and start up the Docker containers docker-compose -f docker-compose-dev.yaml up --build -d
Set Up Account App and Syncing with OpenStack Keystone
Now that we have the Django project up and running with four containers (Django, PostgreSQL, celery and Redis), let’s move on to the next steps. We'll create an account app and define an account model that will trigger a signal on new user creation to sync the user with OpenStack. The syncing process will be handled asynchronously using Celery.
Create the Account App
- Run the following command on the terminal with your virtual environment still activated to create a new account app
python manage.py startapp account
Add the account app to the INSTALLED_APPS list in the Django project's settings.py
INSTALLED_APPS = [ #new apps 'rest_framework', 'account', ]
Now we have account app folder created - we can proceed to create the model!
Define the Account Model
Edit the models.py file in the account app folder to include the Account model and signals to trigger user sync
from django.db import models
from django.contrib.auth.models import AbstractUser
from django.db.models.signals import post_save
from django.dispatch import receiver
from .tasks import sync_user_to_openstack
class Account(AbstractUser):
"""
Custom user model that extends AbstractUser.
Add any additional fields if needed.
"""
email = models.CharField(max_length=255, unique=True)
open_stack_id = models.CharField(blank=True, max_length=50)
open_stack_token = models.TextField(blank=True)
token_created_at = models.DateTimeField(blank=True, null=True)
token_expires_at = models.DateTimeField(blank=True, null=True)
USERNAME_FIELD = 'email'
REQUIRED_FIELDS = ['password', "username"]
def __str__(self) -> str:
return str(self.email)
# model to track failed celery tasks
class FailedTask(models.Model):
task_id = models.CharField(max_length=255)
exc = models.TextField()
args = models.JSONField()
kwargs = models.JSONField()
einfo = models.TextField()
@receiver(post_save, sender=Account)
def create_user_sync(sender, instance, created, **kwargs):
"""
Signal to sync a newly created user to OpenStack.
"""
if created:
sync_user_to_openstack.delay(instance.id)
Don’t forget to update settings.py file to tell Django about the account model - update settings.py file with the content below
AUTH_USER_MODEL = "account.Account"
OPEN_STACK_AUTH_URL = os.getenv("OPEN_STACK_AUTH_URL")
OPEN_STACK_ADMIN_PROJECT_ID = os.getenv("OPEN_STACK_ADMIN_PROJECT_ID")
OPEN_STACK_ADMIN_USERNAME = os.getenv("OPEN_STACK_ADMIN_USERNAME")
OPEN_STACK_ADMIN_PASSWORD=os.getenv('OPEN_STACK_ADMIN_PASSWORD')
Now let’s create the celery task to handle to background communication to openstack platform. In the account app, create a tasks.py file to define the task for syncing the user to OpenStack
from celery import shared_task from .models import Account from .utils import create_openstack_user from asgiref.sync import async_to_sync class CallbackTask(Task): def on_failure(self, exc, task_id, args, kwargs, einfo): # add the failed task to db for later reference FailedTask.objects.create( task_id=task_id, exc=str(exc), args=args, kwargs=kwargs, einfo=str(einfo) ) # you can also send an alert email @shared_task(bind=True, base=CallbackTask, max_retries=4) def sync_user_to_openstack(self, user_id): try: user_instance = Account.objects.get(id=user_id) async_to_sync(create_openstack_user)(user_instance) except Exception as exc: # Retry the task raise self.retry(exc=exc, countdown=5) # retry in 5 seconds
Create .utils.py file in the account folder and add the content below
import httpx
from django.conf import settings
from .token_mgt import get_openstack_token
from asgiref.sync import sync_to_async
import logging
logger = logging.getLogger("account")
async def create_openstack_user(user_instance):
_,token, _, _ = await get_openstack_token(admin_pass='yes')
# Assemble the user data for OpenStack
user_data = { "user": { "name": user_instance.username, "password": user_instance.password[20:61], "email": user_instance.email, "enabled": True, "options": { "ignore_password_expiry": True } } }
# OpenStack API endpoint
url = f"{settings.OPEN_STACK_AUTH_URL}/v3/users"
# Asynchronously call OpenStack API to create user
async with httpx.AsyncClient(verify=False, timeout=120) as client:
user_response = await client.post(url, json=user_data, headers={"X-Auth-Token": token})
if user_response.status_code != 201:
logger.error(f"Failed to create OpenStack user for {user_instance.username}: {user_response.text}")
openstack_user_id = user_response.json()['user']['id']
# Save the OpenStack user ID to the Django user instance
user_instance.open_stack_id = openstack_user_id
await sync_to_async(user_instance.save, thread_sensitive=True)()
return openstack_user_id
If you look at the “create_openstack_user“ function above, you will notice that we are calling another function named “get_openstack_token“ - what’s it about?
To create a new user on Openstack via API, We need to use an already generated valid admin token. This can be by following the steps below
- Create token_mgt file in account app folder and add the content below - The code below allows us to manage user’s token validity either admin or normal user
import json
from datetime import datetime, timedelta, timezone
import os
from cryptography.fernet import Fernet
import httpx
from django.conf import settings
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
import base64
import logging
logger = logging.getLogger(__name__)
TOKEN_FILE_PATH = 'openstack_token.json'
def get_fernet_key():
# Django's SECRET_KEY is used here as the source of entropy.
secret_key_bytes = settings.SECRET_KEY.encode()
# Derive a key using HKDF
hkdf = HKDF(
algorithm=hashes.SHA256(),
length=32, # Length required for Fernet keys
salt=None,
info=b'fernet key derivation'
)
key = hkdf.derive(secret_key_bytes)
return base64.urlsafe_b64encode(key) # Fernet requires the key to be base64 encoded
def encrypt_data(data):
key = get_fernet_key()
cipher = Fernet(key)
encrypted_bytes = cipher.encrypt(data.encode())
encrypted_base64 = base64.urlsafe_b64encode(encrypted_bytes)
return encrypted_base64.decode()
def decrypt_data(encrypted_data):
key = get_fernet_key()
cipher = Fernet(key)
encrypted_bytes = base64.urlsafe_b64decode(encrypted_data.encode())
decrypted_bytes = cipher.decrypt(encrypted_bytes).decode()
return decrypted_bytes
def save_token(token, expires_at):
token_data = {
'token': encrypt_data(token),
'created': datetime.now().isoformat(),
'expires': expires_at
}
with open(TOKEN_FILE_PATH, 'w') as f:
json.dump(token_data, f, indent=4)
def load_token():
if not os.path.exists(TOKEN_FILE_PATH):
return None
try:
with open(TOKEN_FILE_PATH, 'r') as f:
token_data = json.load(f)
except json.decoder.JSONDecodeError:
return False
return token_data
def token_is_valid(expires_at, is_admin=None):
if is_admin:
expires_at = datetime.fromisoformat(expires_at)
# Get the current time in UTC
current_utc_time = datetime.now(timezone.utc)
# Add a 5-minute buffer to the expiration time
if current_utc_time >= expires_at - timedelta(minutes=5):
return False
return True
async def get_new_token_or_use_old_one(user=None, admin_pass=None):
if admin_pass:
username, password = settings.OPEN_STACK_USERNAME, settings.OPEN_STACK_PASSWORD
token_data = load_token()
if not token_data:
token = None
else:
expires_at, token = token_data.get('expires'), token_data.get('token')
if not expires_at or not token:
token = None
elif user:
token, expires_at = user.open_stack_token, user.token_expires_at
username, password = user.username, user.password[20:61]
if token:
if token_is_valid(expires_at, admin_pass):
return (None, decrypt_data(token), None, None)
return (username, password)
async def get_openstack_token(user=None, admin_pass=None):
get_new_token_check = await get_new_token_or_use_old_one(user, admin_pass)
if len(get_new_token_check) == 4:
return get_new_token_check[0], get_new_token_check[1], get_new_token_check[2], get_new_token_check[3]
else:
username, password = get_new_token_check[0], get_new_token_check[1]
auth_url = f"{settings.OPEN_STACK_AUTH_URL}/v3/auth/tokens"
auth_data = {
"auth": {
"identity": {
"methods": ["password"],
"password": {
"user": {
"name": username,
"domain": {"id": "default"},
"password": password
}
}
}
}
}
if admin_pass:
auth_data['auth'].update({ "scope": {
"project": {
"id": settings.OPEN_STACK_ADMIN_PROJECT_ID
}
}})
headers = {'Content-Type': 'application/json'}
async with httpx.AsyncClient(verify=False) as client:
resp = await client.post(auth_url, json=auth_data, headers=headers)
if resp.status_code != 201:
from rest_framework import serializers
raise serializers.ValidationError("Error logging in to open stack")
token = resp.headers["X-Subject-Token"]
token_body = resp.json() # Parse the JSON response body
expires_at = token_body['token']['expires_at']
if admin_pass:save_token(token, expires_at)
return encrypt_data(token), token, resp.json()['token']['expires_at'], resp.json()['token']['issued_at']
Go into the docker container and run the migrations commands to apply the changes to the database
python manage.py makemigrations account
python manage.py migrate
User Registration and Login Endpoints
It’s time to use django rest framework installed earlier to create endpoints within our project. We will only asked users for email and password to register.
Please follow the steps below
Create urls.py file in account app folder and add the registration url below
from django.urls import path from . import views urlpatterns = [ path('register', views.registration, name = 'registration'), ]In views.py file in account app folder add the registration view as below to communicate with serializer and save the user into the database
from rest_framework.decorators import api_view from .serializers import RegistrationSerializer from rest_framework.response import Response from rest_framework import status @api_view(['POST']) def registration(request): serializer = RegistrationSerializer(data = request.data) serializer.is_valid(raise_exception=True) serializer.save() return Response({"success": "User registered successfully"}, status=status.HTTP_201_CREATED)Create serializers.py file in the account app folder to handle request data and response serialization
from rest_framework import serializers from .models import Account import re class RegistrationSerializer(serializers.ModelSerializer): class Meta: model = Account fields = ["email", "password"] def validate_password(self, value): if len(value) < 8: raise serializers.ValidationError("This password is too short. It must contain at least 8 characters.", 400) if value.isalpha() or value.isnumeric(): raise serializers.ValidationError("password must be alphanumeric", 400) if not re.search(r"[^a-zA-Z0-9]", value): raise serializers.ValidationError("Password must contain at least one special character.", 400) def validate_emailadd(self, value): if Account.objects.filter(email=value).exists(): raise serializers.ValidationError("This email is already in use.", 400) return value def create(self, validated_data): # Create the user instance user = Account( email=validated_data['email'], username=validated_data['email'] ) # Set the password using Django's make_password user.set_password(validated_data['password']) # Save the user instance user.save() return userupdate the urls.py file in the openstack project folder to know about the existence of urls in the account app
from django.contrib import admin from django.urls import path, include # new line urlpatterns = [ path('admin/', admin.site.urls), path("account/", include('account.urls')) #new line ]
It's time to test the endpoint — restart the containers and make the API call. I used Postman as shown below.
This article is already long, so we'll cover logging users in the next post. Until then, take care and see you soon!!!