DeepSeek releases DeepInfer inference framework as open source

DeepSeek has made DeepInfer, its internally developed inference framework, available as open-source software. The company says the framework runs its production services, and the released code supports DeepSeek-V4.1-Flash.

The software includes optimized computation routines, distributed processing and speculative decoding, which drafts output before checking it. It also supports structured responses and repeatable sampling, and carries an Apache 2.0 license.

DeepSeek says sharing its deployment technology is intended to reduce computing costs and energy consumption. Those are stated goals rather than a guarantee of equivalent savings for every deployment.

The public package supports offline inference, but developers must supply additional infrastructure to operate a complete service. DeepSeek lists missing components including a cache system and load balancing, making the release a foundation for deployment rather than a complete hosted-service replacement.