Streaming video and audio in torchaudio

Hi,

I’m trying to use torchaudio.io.StreamReader(), followed by add_basic_video_stream and add_basic_audio_stream, to stream video and audio from the webcam/microphone simultaneously on Linux.

There are a few tutorials out there on how to do this on Windows or Mac, but there seems to be none for Linux. Specifically, I’m confused about what I should select as format= and src= in StreamReader(). Through a bit of trial and error, I have figured out that I can use src=/dev/video0 and format=v4l2 to stream video, but can’t figure out what src and format I should use to have both streams - video and audio - simultaneously.

Any help would be greatly appreciated :slight_smile:

I suppose the first question here would be how to stream audio using StreamReader, i.e., which src and format I should use (can’t seem to figure this out and there seem to be no examples online for Linux), and then the second question would be how to combine the video and audio srcs and formats.

I had this issue as well. First, I did a pacmd list sources to find my device card. I have a 6-channel array on card 4 that samples at 16000 Hz.

To get the StreamReader started, I did the following:
streamer = torchaudio.io.StreamReader(src="hw:4, format="alsa", option={"sample_rate":"16000","channels":"6"})

The actual option names come directly from the FFMPEG alsa options here: FFmpeg Devices Documentation