在 Android 中重定向音频/创建备用声音路径

Redirecting audio / creating alternate sound paths in Android

提问by jpalm

Does anyone have experience (using OpenSL ES, ALSA, etc.) with redirecting audio or creating new sound paths in Android? The end goal is to create a virtual microphone to replace the external microphone, where one can play audio files as if they were speaking into the microphone. Applications accessing the microphone with AudioSource.MICshould use this alternate stream. It's not necessary for it to work with voice calls, I believe achieving that sort of functionality is harder as it's all done within the radio.

有没有人有在 Android 中重定向音频或创建新声音路径的经验（使用 OpenSL ES、ALSA 等）？最终目标是创建一个虚拟麦克风来取代外部麦克风，在那里人们可以播放音频文件，就像他们对着麦克风说话一样。访问麦克风的应用程序AudioSource.MIC应使用此备用流。它没有必要与语音通话一起工作，我相信实现这种功能更难，因为这一切都是在无线电中完成的。

Any ideas on where to begin? I've done some research with OpenSL and ALSA, but it looks like I'll need to package new firmware (ROM) in order to define custom audio paths. If it can be avoided I'd like to create an application-level solution. The phones are 'rooted' (have su binaries). The target device for this is the Samsung Galaxy S4 Google Edition (GT-i9505G). Specifically I'm looking for audio driver configurations / source code or any references for the i9505G.

关于从哪里开始的任何想法？我已经对 OpenSL 和 ALSA 进行了一些研究，但看起来我需要打包新固件 (ROM) 才能定义自定义音频路径。如果可以避免，我想创建一个应用程序级解决方案。电话是“扎根”的（有 su 二进制文件）。此操作的目标设备是三星 Galaxy S4 Google 版 (GT-i9505G)。具体来说，我正在寻找 i9505G 的音频驱动程序配置/源代码或任何参考资料。

Thanks in advance!

提前致谢！

edit - I've checked out the CyanogenMod 10.2 source tree, along with the jfltexx drivers and kernel. Here are the contents of kernel/samsung/jf/sound: http://pastebin.com/7vK8THcZ. Is this documented anywhere?

编辑 - 我已经检查了 CyanogenMod 10.2 源代码树，以及 jfltexx 驱动程序和内核。以下是 kernel/samsung/jf/sound 的内容：http: //pastebin.com/7vK8THcZ。这在任何地方都有记录吗？

回答by Michael

I once implemented the functionality you're after on a phone based on Qualcomm's APQ8064 platform (which seems to be nearly the same platform as the one in your target device). Below is a summary of what I can recall from this, as I no longer have access to the code I wrote, or an environment where I can easily do these kinds of modifications. So if this answer reads like a mess of fragmentary memories, that's because that's exactly what it is.

我曾经在基于 Qualcomm 的 APQ8064 平台（这似乎与目标设备中的平台几乎相同的平台）的手机上实现了您所追求的功能。以下是我从中可以回忆起的内容的摘要，因为我无法再访问我编写的代码，也无法访问可以轻松进行此类修改的环境。所以如果这个答案读起来像是一堆零碎的记忆，那是因为它就是这样。

This info may also apply more-or-less to other Qualcomm platforms (like the MSM8960 or MSM8974), but will most likely be completely useless for platforms from other vendors (NVidia Tegra, Samsung Exynos, TI OMAP, etc).

此信息也可能或多或少适用于其他高通平台（如 MSM8960 或 MSM8974），但很可能对其他供应商的平台（NVidia Tegra、三星 Exynos、TI OMAP 等）完全无用。

A brief note: The method I used means that the audio that the recording application gets will have gone through mixing / volume control in the Android multimedia framework and/or the platform's multimedia DSP. So if you're playing something at 75% volume, recording it, and then playing back the recording at 75% volume it might end up sounding pretty quiet. If you want to get unprocessed PCM data (after decoding, but before mixing / volume control) you'll have to look into some other approach, e.g. customizing the AudioFlinger, but that's not something I've tried or can provide info on.

简短说明：我使用的方法意味着录音应用程序获取的音频将经过 Android 多媒体框架和/或平台的多媒体 DSP 中的混音/音量控制。因此，如果您以 75% 的音量播放某些内容，录制它，然后以 75% 的音量播放录音，最终听起来可能会非常安静。如果您想获得未处理的 PCM 数据（在解码之后，但在混音/音量控制之前），您将不得不研究其他一些方法，例如自定义AudioFlinger.

---

---

A few locations of interest:

几个感兴趣的地方：

The platform's audio drivers. Particularly the msm-pcm-routing.c file.

平台的音频驱动程序。特别是msm-pcm-routing.c 文件。

The ALSA UCM (Use-Case Manager) settings file. This is just an example UCM settings file. There are many variants of these files depending on the exact platform used, so your's may have a slightly different name (though it shouldstart with snd_soc_msm_), and its contents will probably also differ slightly from the one I linked to.
NOTE for Kitkat and later:The UCM settings files were used on Jellybean (and possibly ICS). My understanding is that these settings have been moved to a file named mixer_paths.xmlon Kitkat. The contents are pretty much the same, just in a different format.

ALSA UCM（用例管理器）设置文件。这只是一个示例 UCM 设置文件。根据所使用的确切平台，这些文件有许多变体，因此您的名称可能略有不同（尽管它应该以开头snd_soc_msm_），并且其内容也可能与我链接的内容略有不同。
Kitkat 及更高版本的注意事项：UCM 设置文件用于 Jellybean（可能还有 ICS）。我的理解是这些设置已移至名为mixer_paths.xmlKitkat的文件中。内容几乎相同，只是格式不同。

The audio HAL code. The ALSA UCM is present in libalsa-intf, and the AudioHardware/ AudioPolicyManager/ ALSADevicecode is present in audio-alsa. Note that this code is for Jellybean, since that's the lastest version that I'm familiar with. The directory structure (and possibly some of the files / classes) differs on Kitkat.

音频 HAL 代码。的ALSA UCM存在于libalsa-intf，和AudioHardware/ AudioPolicyManager/ALSADevice代码的存在audio-alsa。请注意，此代码适用于 Jellybean，因为这是我熟悉的最新版本。Kitkat 上的目录结构（可能还有一些文件/类）有所不同。

If you open up the UCM settings file and search for "HiFiPROXY Rx"you'll find something like this:

如果您打开 UCM 设置文件并搜索，"HiFiPROXY Rx"您会发现如下内容：

SectionVerb
Name "HiFiPROXY Rx"

EnableSequence
    'AFE_PCM_RX Audio Mixer MultiMedia1':1:1
EndSequence

DisableSequence
    'AFE_PCM_RX Audio Mixer MultiMedia1':1:0
EndSequence

# ALSA PCMs
CapturePCM 0
PlaybackPCM 0
EndSection

This defines a verb(essentially the basis of an audio use-case; there are also modifiersthat can be applied on top of verbs for stuff like simultaneous playback and recording) with the name "HiFiPROXY Rx"(the HiFimoniker is used for most non-voice-call verbs, PROXYrefers to the audio device used, and Rxmeans output) and specifies which ALSA control(s) to write to, and what to write to them, when the use-case should be enabled / disabled. Finally it lists the ALSA PCM playback / capture devices to use in this use-case. For example, PlaybackPCM 0means that playback device 0 should be used (the ALSA card is implied to be the one that represents the built-in hardware codec, which typically is card 0). These verbs are selected by the audio HAL based on the use-case (music playback, voice call, recording, ...), what accessories you've got attached, etc.

这定义了一个动词（基本上是音频用例的基础；还有修饰符可以应用于动词之上，用于同步播放和录音之类的东西）和名称"HiFiPROXY Rx"（HiFi名字用于大多数非语音呼叫动词，PROXY指的是使用的音频设备，并Rx表示输出）并指定要写入的 ALSA 控件以及写入的内容，何时应启用/禁用用例。最后，它列出了在此用例中使用的 ALSA PCM 播放/捕获设备。例如，PlaybackPCM 0意味着应该使用播放设备 0（ALSA 卡被暗示为代表内置硬件编解码器的卡，通常是卡 0）。这些动词由音频 HAL 根据用例（音乐播放、语音通话、录音等）、您附加的附件等选择。

---

---

If you look up "AFE_PCM_RX Audio Mixer"in the msm_qdsp6_widgets tablein msm-pcm-routing.cyou'll see that it refers to a list of mixer controls named afe_pcm_rx_mixer_controlsthat looks like this:

如果您"AFE_PCM_RX Audio Mixer"在msm_qdsp6_widgets 表中查找，msm-pcm-routing.c您会看到它指的是一个名为的混音器控件列表，afe_pcm_rx_mixer_controls如下所示：

static const struct snd_kcontrol_new afe_pcm_rx_mixer_controls[] = {
    SOC_SINGLE_EXT("MultiMedia1", MSM_BACKEND_DAI_AFE_PCM_RX,
    MSM_FRONTEND_DAI_MULTIMEDIA1, 1, 0, msm_routing_get_audio_mixer,
    msm_routing_put_audio_mixer),
    SOC_SINGLE_EXT("MultiMedia2", MSM_BACKEND_DAI_AFE_PCM_RX,
    ... and so on...

This lists the front end DAIs that you are allowed to connect to the back end DAI (AFE_PCM_RX). To get an idea of how these relate to one another, see these diagrams.
AFE_PCM_RXand AFE_PCM_TXis a pair of DAIs on some of Qualcomm's platforms that implement a sort of dummy/proxy device. What you do is feed audio into AFE_PCM_RXwhich then gets processed by the multimedia DSP (QDSP), and then you can read it back through AFE_PCM_TX. This is used to implement USB and WiFi audio routing, and also A2DP IIRC.

这列出了允许您连接到后端 DAI ( AFE_PCM_RX) 的前端 DAI 。要了解这些之间的关系，请参阅这些图表。
AFE_PCM_RX并且AFE_PCM_TX是在一些实现一种虚拟/代理设备的高通平台的一对讲台。您所做的是将音频输入AFE_PCM_RX其中，然后由多媒体 DSP (QDSP) 处理，然后您可以通过AFE_PCM_TX. 这用于实现 USB 和 WiFi 音频路由，以及 A2DP IIRC。

Back to the AFE_PCM_RX Audio Mixer MultiMedia1line: This says that you're feeding MultiMedia1into the AFE_PCM_RX Audio Mixer. MultiMedia1is used for normal playback/recording, and corresponds to pcmC0D0(you should be able to list the devices on your phone with adb shell cat /proc/asound/devices). There are other front end DAIs, like MultiMedia3and MultiMedia5that are used in special cases like low-latency playback and low-power audio playback.
When you feed MultiMedia1to the AFE_PCM_RX Audio Mixereverything you write to playback device 0 on card 0 will be fed into the AFE_PCM_RXback end DAI. To read it back you could set up a UCM verb that does something like 'MultiMedia1 Mixer AFE_PCM_TX':1:1, and then you'd read from pcmC0D0c(which should be the default ALSA capture device).

回到这AFE_PCM_RX Audio Mixer MultiMedia1条线：这表示您正在MultiMedia1向AFE_PCM_RX Audio Mixer. MultiMedia1用于正常播放/录音，对应pcmC0D0（你应该可以用 列出你手机上的设备adb shell cat /proc/asound/devices）。还有其他前端 DAI，例如MultiMedia3和MultiMedia5用于低延迟播放和低功耗音频播放等特殊情况。
当您MultiMedia1将AFE_PCM_RX Audio Mixer写入到卡 0 上的播放设备 0的所有内容输入到AFE_PCM_RX后端 DAI 中时。要回读它，您可以设置一个执行类似 的 UCM 动词，'MultiMedia1 Mixer AFE_PCM_TX':1:1然后您可以从中读取pcmC0D0c（这应该是默认的 ALSA 捕获设备）。

---

---

A simple test would be to pull the UCM settings file from your phone (should be located somewhere under /system/etc/) and amend the "HiFi"verb's EnableSequencewith something like:

一个简单的测试是从您的手机中提取 UCM 设置文件（应位于 下的某处/system/etc/）并使用以下内容修改"HiFi"动词EnableSequence：

'AFE_PCM_RX Audio Mixer MultiMedia1':1:1
'AFE_PCM_RX Audio Mixer MultiMedia3':1:1
'AFE_PCM_RX Audio Mixer MultiMedia5':1:1

(and similarly in the DisableSequence, but with :1:0at the end of each line).

（同样在DisableSequence, 但:1:0在每行的末尾）。

Then go to the "Capture Music"modifier (this is the poorly named modifier for normal recording) and change SLIM_0_TXto AFE_PCM_TX.

然后转到"Capture Music"修饰符（这是普通录音的名称不佳的修饰符）并更改SLIM_0_TX为AFE_PCM_TX.

Copy your modified UCM settings file back to the phone (requires root permission), and reboot the phone. Then start some playback (have a wired headset/headphone attached, and disable touch sounds so that the low-latency verb doesn't get selected), and start a recording from AudioSource.MIC. Afterwards, check the recording and see if you were able to record the playback audio. If not, then perhaps the low-power audio verb was selected and you'll have to modify the "HiFi Low Power"verb similarly to what you did with the "HiFi"verb. It will help you if you have all the debug prints enabled in the audio HAL (i.e. uncomment #define LOG_NDEBUG 0in all the cpp files where you can find it) so that you can see which UCM verbs / modifiers that get selected.

将修改后的 UCM 设置文件复制回手机（需要 root 权限），然后重启手机。然后开始一些播放（连接有线耳机/耳机，并禁用触摸声音，以便不会选择低延迟动词），并从AudioSource.MIC. 之后，检查录音，看看您是否能够录制播放音频。如果不是，那么可能选择了低功率音频动词，您必须像修改"HiFi Low Power"动词一样修改"HiFi"动词。如果您在音频 HAL 中启用了所有调试打印（即#define LOG_NDEBUG 0在您可以找到的所有 cpp 文件中取消注释），它将对您有所帮助，以便您可以看到哪些 UCM 动词/修饰符被选中。

---

---

The modification I described above gets a bit tedious since you have to cover all the MultiMediafront end DAIs for all relevant verbs and modifiers.
IIRC, I was able to simplify this into just a single line per verb/modifier:

我上面描述的修改有点乏味，因为您必须涵盖MultiMedia所有相关动词和修饰符的所有前端 DAI。
IIRC，我能够将其简化为每个动词/修饰符仅一行：

'AFE_PCM_RX Port Mixer SLIM_0_RX':1:1

If you look at the "HiFi", "HiFi Low Power", "HiFi Lowlatency"verbs you'll see that they all use the SLIMBUS_0_RXback end DAI, so I'm taking advantage of that by using the AFE_PCM_RX Port Mixerwhich lets me set up a connection from a back end DAI to another back end DAI. If you look at the afe_pcm_rx_port_mixer_controlsand intercontables in msm-pcm-routing.cyou'll notice that there's no SLIM_0_RXentry for AFE_PCM_RX Port Mixer, so you'd have to add those yourself (it's just a matter of copy-pasting some of the existing lines and changing the names).

如果您查看"HiFi", "HiFi Low Power","HiFi Lowlatency"动词，您会发现它们都使用SLIMBUS_0_RX后端 DAI，因此我通过使用AFE_PCM_RX Port Mixer它来利用它，这让我可以建立从后端 DAI 到另一个后端的连接戴。如果您查看afe_pcm_rx_port_mixer_controls和 中的intercon表，msm-pcm-routing.c您会注意到 没有SLIM_0_RX条目AFE_PCM_RX Port Mixer，因此您必须自己添加这些条目（这只是复制粘贴一些现有行并更改名称的问题）。

---

---

Some of the other changes you'd probably have to make:

您可能需要进行的其他一些更改：

• In frameworks/baseand frameworks/av(e.g. AudioManager, AudioService, AudioSystem) you'd have to add a new AudioSourceconstant and make sure that it gets recognized in all the necessary places.

• In the UCM settings file you'd have to add some new verbs / modifiers to set up the ALSA controls properly when your new AudioSourceis used.

• In the audio HAL you'd have to make some changes so that your new verbs / modifiers get selected when your new AudioSourceis used. Note that there's a base class of AudioPolicyManagerALSAcalled AudioPolicyManagerBasewhich you also might have to modify (it's located elsewhere in the source tree).

• 在frameworks/base和frameworks/av（例如AudioManager, AudioService, AudioSystem）中，您必须添加一个新AudioSource常量并确保它在所有必要的地方都能被识别。

• 在 UCM 设置文件中，您必须添加一些新的动词/修饰符，以便在AudioSource使用新的时正确设置 ALSA 控件。

• 在音频 HAL 中，您必须进行一些更改，以便在使用新动词/修饰语时选择新动词/修饰符AudioSource。请注意，有一个AudioPolicyManagerALSA被调用的基类AudioPolicyManagerBase，您可能还需要对其进行修改（它位于源代码树的其他位置）。