Overview
Since build 5.2.795 it is possible to extract raw audio data from published stream in PCM 16 bit format with a following processing on server side. For example, stream sound can be recorded to a file.
Note that the stream published should have at least one subscriber to decode and extract audio from it.
Audio processing implementation
To intercept raw audio data, a Java class implementing IDecodedPcmInterceptor interface should be developed. The method pcmDecoded() of this class will receive decoded audio packets in PCM format, as byte array. Let's take a look to class implementation example to record raw audio from a stream into a WAV file:
package com.flashphoner.pcmInterceptor;
// Import Flashphoner SDK packages as needed
import com.flashphoner.media.rtp.recorder.OutputFileType;
import com.flashphoner.media.utils.FileNameUtils;
import com.flashphoner.media.utils.WaveUtil;
import com.flashphoner.sdk.media.IDecodedPcmInterceptor;
import com.flashphoner.sdk.setting.Settings;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
// Import standard Java packages as needed
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.Timer;
import java.util.TimerTask;
/**
* Custom decoded audio interceptor implementation example
* The example records first 10 seconds of audio track from a stream published to WAV file
*/
public class DecodedPcmInterceptorTest implements IDecodedPcmInterceptor{
// Will log errors to server log.
private static final Logger log = LoggerFactory.getLogger("DecodedPcmInterceptorTest");
// File object to write data
protected RandomAccessFile incomingRecorder;
// Last audio packet timestamp
private volatile long lastTs;
// Sampling rate to write to a file header
private int samplingRate;
// Number of cahnnels to write to a file header
private int numChannels;
// Timer to stop recording
private Timer cancelTimer;
// Stream name to form file name
private String streamName;
public DecodedPcmInterceptorTest() {
}
/**
* Method to handle decoded audio
* @param streamName - stream name
* @param pcm - decoded audio data from packet as byte array
* @param samplingRate - audio track sampling rate
* @param numChannels - audio track number of channels
* @param timestamp - audio packet timestamp
*/
@Override
public void pcmDecoded(String streamName, byte[] pcm, int samplingRate, int numChannels, long timestamp) {
// Remember data to write to the file header
updateSr(samplingRate);
updateNumChannels(numChannels);
// Remember the stream name and create the file to write
updateStreamName(streamName);
// Start timeout to stop recording
startCloseTimer();
try {
// Write audio data to the file
recordIncoming(pcm, timestamp);
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* Method to start timer task which should stop recording after 10 seconds
*/
public void startCloseTimer() {
if (cancelTimer == null) {
cancelTimer = new Timer();
cancelTimer.schedule(new TimerTask() {
@Override
public void run() {
try {
close();
} catch (IOException e) {
e.printStackTrace();
}
}
}, 10000);
}
}
/**
* Method to store audio sampling rate to write it to the file header
*/
public void updateSr(int samplingRate) {
if (this.samplingRate == 0) {
this.samplingRate = samplingRate;
}
}
/**
* Method to store audio channels number to write it to the file header
*/
public void updateNumChannels(int numChannels) {
if (this.numChannels == 0) {
this.numChannels = numChannels;
}
}
/**
* Method to store the stream name and create the file to write
*/
public void updateStreamName(String streamName) {
if (this.streamName == null || this.streamName.isEmpty()) {
// Create the file name
this.streamName = FileNameUtils.adaptRecordName(streamName+".wav");
try {
// Create the file and reserve header space
incomingRecorder = new RandomAccessFile(Settings.RECORD.getValue()+"/"+this.streamName, "rw");
incomingRecorder.write(new byte[OutputFileType.WAV_HEADER_OFFSET]);
} catch (IOException e) {
log.error("Can't create DecodedPcmInterceptorTest, " + e.getMessage());
}
log.info("Create DecodedPcmInterceptorTest");
}
}
/**
* Method to write audio data to the file
*/
public void recordIncoming(byte[] data, long ts) throws IOException {
incomingRecorder.write(data);
lastTs = ts;
}
/**
* Method to close file
*/
private void close() throws IOException {
// Write header to the file
writeHeader();
try {
// Close the file
incomingRecorder.close();
} catch (IOException e) {
e.printStackTrace();
}
log.info("Close DecodedPcmInterceptorTest");
}
/**
* Method to write header to the beginning of the file
*/
protected void writeHeader() throws IOException {
// Get the file size
int size = (int) incomingRecorder.length();
// Form the header
byte[] header = WaveUtil.getPcmWaveHeader((size - OutputFileType.WAV_HEADER_OFFSET), samplingRate, numChannels);
// Write the header to the beginning of the file
incomingRecorder.seek(0);
incomingRecorder.write(header);
}
}
Then the class should be complied into byte code. To do this, create folder tree accordind to class package name
mkdir -p com/flashphoner/pcmInterceptor
and execute the command
javac -cp /usr/local/FlashphonerWebCallServer/lib/wcs-core.jar:/usr/local/FlashphonerWebCallServer/lib/slf4j-api-1.6.4.jar ./com/flashphoner/pcmInterceptor/DecodedPcmInterceptorTest.java
Now, pack the code compiled to jar file
jar -cf testPcmInterceptor.jar ./com/flashphoner/pcmInterceptor/*.class
and copy this file to WCS libraries folder
cp testPcmInterceptor.jar /usr/local/FlashphonerWebCallServer/lib
To use custom frames interceptor class, set its package name and folder to record WAV files to the following parameters in flashphoner.properties file
decoded_pcm_interceptor=com.flashphoner.pcmInterceptor.DecodedPcmInterceptorTest record=/usr/local/FlashphonerWebCallServer/records
and restart WCS.
A separate folder for custom Java libraries
Since build 5.2.1512, custom layout Java libraries (jar files) should be placed to the folder /usr/local/FlashphonerWebCallServer/lib/custom
cp testPcmInterceptor.jar /usr/local/FlashphonerWebCallServer/lib/custom
This folder is kept while updating WCS to a newer builds. A jar files do not need to be copied again after updating.
Testing
1. Publish a test stream in Two Way Streaming example https://test1.flashphoner.com:8444/client2/examples/demo/streaming/two_way_streaming/two_way_streaming.html, where test1.flashphoner.com is WCS server address
2. Check if WAV file exists in /usr/local/FlashphonerWebCallServer/records/ folder

