Announcement: PyTorch - Blender integration

Training with artificial images is becoming increasingly important to address the lack of real data sets in various niche areas. Yet, many today’s approaches write 2D/3D simulations from scratch.

To improve this situation and make better use of existing pipelines, we’ve been working towards an integration between Blender, an open-source real-time physics enabled animation software, and PyTorch. Today we announce blendtorch, an open-source Python library that seamlessly integrates distributed Blender renderings into PyTorch data pipelines at 60FPS (640x480 RGBA).

– Christoph

13 Likes

That looks very cool. Thank you for sharing!

Thomas

2 Likes

This looks really cool! :slight_smile:

1 Like

@ptrblck and @tom thanks for your feedback. We have just released v0.2 which brings an even better PyTorch integration, plus OpenAI.gym support that allows us to use Blender as modelling, simulation and interactive manipulation framework to train your RL agents through the well known OpenAI gym interface. Below is an animated gif showing a recreation of the cartpole demo running in Blender.

2 Likes

Hi @Christoph_Heindl !

Your project looks really interesting…

I am trying to create a chatbot using Blender and Pytorch. I’ve got the blend file in which I’ve created the poses (the rain rig). The Rain rig responds to the audio input based on the (.dat) file. And honestly, I’m still clueless about how it gonna create poses based on output from the conversational module in real-time.

Although I came across blendtorch- a Python framework to seamlessly integrate Blender into PyTorch datasets for deep learning from artificial visual data

But I’m still wondering how it will create poses on the go ie. based on conversational input in real-time. Can you please guide me a little on how shall I proceed?

I’m trying to create something like this:

  1. GitHub - huihut/Facemoji: 😆 A voice chatbot that can imitate your expression. OpenCV+Dlib+Live2D+Moments Recorder+Turing Robot+Iflytek IAT+Iflytek TTS or just like digital human Fatema who answers customers’ queries.

Thanks in advance.

Hey @Devendra_Sharma , I was also working on a chatbot in Blender as well, for about a month now.

The chatbot’s simple 3d mechanics are done, just need to integrate the conversational ai component now.

Any luck importing any torch conversation lib so far into blender?

This BlendTorch project looks promising. I love It.

#b3d #ai #NextGen