Python C Library Function Binding Using ctypes

Introduction

Python is a programming language that has close relationship with C. Calling C functions from Python is straightforward via ctypes.

In this blog post, I would like to discuss the usage of ctypes for calling functions from C libraries.

Python C/C++ Library Binding

C Function Binding

When the C function only involves primitive C types, the function binding is extremely simple. In this case, we called memory copy functions from CUDA Runtime library libcudart and copied memory from host to device and from device to host.

libcuda.py

import os
import ctypes
import numpy as np
from ctypes.util import find_library


def find_cudart(root_path: str = os.path.abspath(os.sep),
                cudart_file_name: str = "libcudart.so") -> str:

    for root, dirs, files in os.walk(root_path):
        if cudart_file_name in files:
            return os.path.join(root, cudart_file_name)

    return None


if __name__ == "__main__":

    lib_cudart_file_path = find_library("libcudart")
    if lib_cudart_file_path is None:
        lib_cudart_file_path = find_cudart(root_path=os.path.abspath(os.sep),
                                           cudart_file_name="libcudart.so")
    lib_cudart = ctypes.CDLL(lib_cudart_file_path)

    num_bytes = 16
    num_floats = num_bytes // 4

    # Create nullptr for device memory.
    d_ptr = ctypes.c_void_p(None)

    # Create host buffer.
    input_array = np.ascontiguousarray(
        np.array([8.88] * num_floats).astype(np.float32))
    output_array = np.ascontiguousarray(
        np.array([0.0] * num_floats).astype(np.float32))

    # Input and output buffers are not equal.
    assert not np.array_equal(input_array, output_array)

    # Create pointers to host buffer.
    h_ptr_input = ctypes.c_void_p(input_array.ctypes.data)
    h_ptr_output = ctypes.c_void_p(output_array.ctypes.data)

    # Allocate buffer on GPU.
    status = lib_cudart.cudaMalloc(ctypes.byref(d_ptr),
                                   ctypes.c_size_t(num_bytes))
    assert status == 0

    # Buffer copy from host to device and from device to host.
    # https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1g18fa99055ee694244a270e4d5101e95b
    status = lib_cudart.cudaMemcpy(d_ptr, h_ptr_input,
                                   ctypes.c_size_t(num_bytes), ctypes.c_int(1))
    assert status == 0
    status = lib_cudart.cudaMemcpy(h_ptr_output, d_ptr,
                                   ctypes.c_size_t(num_bytes), ctypes.c_int(2))
    assert status == 0

    # Free buffer on GPU.
    status = lib_cudart.cudaFree(d_ptr)
    assert status == 0

    # Input and output buffers are equal now.
    assert np.array_equal(input_array, output_array)

C Structure Binding

In some scenarios, we will have to use non-primitive structures in C. Then we will have to create the corresponding structures in Python as well.

The following C code is the source code for a custom library libpoint which contains the C structure and function for Point.

point.c

/*
// Convert library code to Object file
gcc -c -o point.o point.c

// Create shared .SO library
gcc -shared -o libpoint.so point.o
*/

struct Point
{
    int x;
    int y;
};

int add(const struct Point* p_1, const struct Point* p_2, struct Point* p_3)
{
    p_3->x = p_1->x + p_2->x;
    p_3->y = p_1->y + p_2->y;

    return 0;
}

To build the shared library for libpoint, please run the following commands.

$ gcc -c -o point.o point.c
$ gcc -shared -o libpoint.so point.o

In the Python script, we created the corresponding Point class and used the pointers to objects for function calls.

libpoint.py

import os
import ctypes
import pathlib


class Point(ctypes.Structure):

    _fields_ = [
        ("x", ctypes.c_int),
        ("y", ctypes.c_int),
    ]

    def __eq__(self, p):

        if self.x == p.x and self.y == p.y:
            return True
        else:
            return False


if __name__ == "__main__":

    lib_point_file_path = os.path.join(pathlib.Path().absolute(),
                                       "libpoint.so")
    lib_point = ctypes.CDLL(lib_point_file_path)

    p_1 = Point(1, 2)
    p_2 = Point(3, 4)
    p_3 = Point(0, 0)
    p_4 = Point(4, 6)

    lib_point.add(ctypes.byref(p_1), ctypes.byref(p_2), ctypes.byref(p_3))

    assert p_3 == p_4

C++ Function/Class Binding

ctypes were developed for supporting C libraries, its support for C++ functions and classes are very limited. If the function and class method only returns pointers instead of custom objects, it is possible to use ctypes for C++ function and class bindings. Please check the Stack Overflow thread for this. However, if the return type is custom objects, we cannot use ctypes for binding the C++ functions and classes. The interface will have to be re-designed.

References

• Python Bindings: Calling C or C++ From Python