Tencentncnn is a neural network inference framework for deploying deep learning models on mobile, embedded, and desktop devices. It is a pure C++ implementation with no third-party runtime dependencies and no BLAS or NNPACK requirement, exposes C and Python APIs, and runs on CPU backends tuned for ARM NEON and multi-core scheduling or on Vulkan GPUs. It is used in Tencent applications including QQ, Qzone, WeChat, and Pitu.
The recommended workflow is PyTorch to pnnx to ncnn: pnnx exports a PyTorch or ONNX model into model.ncnn.param and model.ncnn.bin files, which the ncnn Net class loads for extraction of named outputs. Converters for older formats remain available. Builds target Linux, Windows, macOS, Android, iOS, WebAssembly, Raspberry Pi, NVIDIA Jetson, POWER, RISC-V, and boards such as AllWinner D1 and Loongson 2K1000.
Supported workloads span classification backbones (VGG, ResNet, DenseNet, MobileNet, ShuffleNet, and others), detection and face models (SSD, Faster R-CNN, MTCNN, RetinaFace, YOLOv2 through YOLOv8, YOLOX, NanoDet), segmentation, pose, and OCR (FCN, PSPNet, UNet, YOLACT, SimplePose, PP-OCR), with community projects covering image generation, LLM and vision-language inference, and speech recognition.
Features
- Dependency-free runtime: pure C++ with C API and Python binding; no BLAS or NNPACK
- CPU optimization: ARM NEON kernels and multi-core scheduling for mobile and embedded processors
- Vulkan GPU acceleration: GPU inference on supported platforms
- Memory design: explicit blob and workspace allocators for a small footprint
- Graph support: multi-input, multi-output, and multi-branch networks
- Model conversion: pnnx for PyTorch and ONNX, plus legacy converters for Caffe, MXNet, Darknet, TensorFlow, and Keras
- Precision options: fp16 storage and arithmetic paths and int8 quantized inference
- Extensibility: model optimization passes and custom layers
- Direct loading: .param and .bin models can be loaded by memory reference
- Platform builds: Android, iOS, WebAssembly, Jetson, Raspberry Pi, RISC-V, and desktop operating systems
