C++ 如何将虚拟内存地址转换为物理地址？

How to translate a virtual memory address to a physical address?

提问by

In my C++ program (on Windows), I'm allocating a block of memory and can make sure it stays locked (unswapped and contiguous) in physical memory (i.e. using VirtualAllocEx(), MapUserPhysicalPages() etc).

在我的 C++ 程序（在 Windows 上）中，我正在分配一块内存并可以确保它在物理内存中保持锁定（未交换和连续）（即使用 VirtualAllocEx()、MapUserPhysicalPages() 等）。

In the context of my process, I can get the VIRTUAL memory address of that block, but I need to find out the PHYSICAL memory address of it in order to pass it to some external device.

在我的进程上下文中，我可以获得该块的虚拟内存地址， 但我需要找出它的物理内存地址，以便将它传递给某个外部设备。

1.Is there any way I can translate the virtual address to the physical one within my program, in USER mode?

2.If not, I can find out this virtual to physical mapping only in KERNEL mode. I guess it means I have to write a driver to do it...? Do you know of any readily available driver/DLL/API which I can use, that my application (program) will interface with to do the translation?

3.In case I'll have to write the driver myself, how do I do this translation? which functions do I use? Is it mmGetPhysicalAddress()? How do I use it?

4.Also, if I understand correctly, mmGetPhysicalAddress() returns the physical address of a virtual base address that is in the context of the calling process. But if the calling process is the driver, and I'm using my application to call the driver for that function, I'm changing contexts and I am no longer in the context of the app when the mmGetPhysicalAddress routine is called... so how do I translate the virtual address in the application (user-mode) memory space, not the driver?

1.有没有什么办法可以在我的程序中在用户模式下将虚拟地址转换为物理地址？

2.如果没有，我只能在 KERNEL 模式下找到这个虚拟到物理的映射。我想这意味着我必须编写一个驱动程序才能做到这一点......？您是否知道我可以使用的任何现成的驱动程序/DLL/API，我的应用程序（程序）将与之交互以进行翻译？

3.如果我必须自己编写驱动程序，我该如何翻译？我使用哪些功能？是mmGetPhysicalAddress()吗？我如何使用它？

4.此外，如果我理解正确，mmGetPhysicalAddress() 返回调用进程上下文中的虚拟基地址的物理地址。但是，如果调用进程是驱动程序，并且我正在使用我的应用程序调用该函数的驱动程序，那么我正在更改上下文，并且当 mmGetPhysicalAddress 例程被调用时，我不再处于应用程序的上下文中......所以如何转换应用程序（用户模式）内存空间中的虚拟地址，而不是驱动程序？

Any answers, tips and code excerpts will be much appreciated!!

任何答案、提示和代码摘录将不胜感激！！

Thanks

谢谢

回答by bk1e

In my C++ program (on Windows), I'm allocating a block of memory and can make sure it stays locked (unswapped and contiguous) in physical memory (i.e. using VirtualAllocEx(), MapUserPhysicalPages() etc).

在我的 C++ 程序（在 Windows 上）中，我正在分配一块内存并可以确保它在物理内存中保持锁定（未交换和连续）（即使用 VirtualAllocEx()、MapUserPhysicalPages() 等）。

No, you can't really ensure that it stays locked. What if your process crashes, or exits early? What if the user kills it? That memory will be reused for something else, and if your device is still doing DMA, that will eventually result in data loss/corruption or a bugcheck (BSOD).

不，您无法真正确保它保持锁定状态。如果您的进程崩溃或提前退出怎么办？如果用户杀死它怎么办？该内存将被重用于其他用途，如果您的设备仍在执行 DMA，最终将导致数据丢失/损坏或错误检查 (BSOD)。

Also, MapUserPhysicalPagesis part of Windows AWE (Address Windowing Extensions), which is for handling more than 4 GB of RAM on 32-bit versions of Windows Server. I don't think it was intended to be used to hack up user-mode DMA.

此外，它MapUserPhysicalPages是 Windows AWE（地址窗口扩展）的一部分，用于在 32 位版本的 Windows Server 上处理超过 4 GB 的 RAM。我不认为它旨在用于破解用户模式 ​​DMA。

1. Is there any way I can translate the virtual address to the physical one within my program, in USER mode?

1. 有没有什么办法可以在我的程序中在用户模式下将虚拟地址转换为物理地址？

There are drivers that let you do this, but you cannot program DMA from user mode on Windows and still have a stable and secure system. Letting a process that runs as a limited user account read/write physical memory allows that process to own the system. If this is for a one-off system or a prototype, this is probably acceptable, but if you expect other people (particularly paying customers) to use your software and your device, you should write a driver.

有一些驱动程序可以让您这样做，但是您无法在 Windows 上从用户模式对 DMA 进行编程，并且仍然拥有稳定和安全的系统。让作为受限用户帐户运行的进程读/写物理内存允许该进程拥有系统。如果这是一次性系统或原型，这可能是可以接受的，但如果您希望其他人（特别是付费客户）使用您的软件和设备，您应该编写一个驱动程序。

2. If not, I can find out this virtual to physical mapping only in KERNEL mode. I guess it means I have to write a driver to do it...?

2. 如果没有，我只能在 KERNEL 模式下找到这个虚拟到物理的映射。我想这意味着我必须编写一个驱动程序才能做到这一点......？

That is the recommended way to approach this problem.

这是解决此问题的推荐方法。

Do you know of any readily available driver/DLL/API which I can use, that my application (program) will interface with to do the translation?

您是否知道我可以使用的任何现成的驱动程序/DLL/API，我的应用程序（程序）将与之交互以进行翻译？

You can use an MDL (Memory Descriptor List)to lock down arbitrary memory, including memory buffers owned by a user-mode process, and translate its virtual addresses into physical addresses. You can also have Windows temporarily create an MDL for the buffer passed into a call to DeviceIoControlby using METHOD_IN_DIRECTor METHOD_OUT_DIRECT.

您可以使用MDL（内存描述符列表）来锁定任意内存，包括用户模式进程拥有的内存缓冲区，并将其虚拟地址转换为物理地址。您还可以让 Windows 临时为DeviceIoControl使用METHOD_IN_DIRECT或传递给调用的缓冲区创建 MDL METHOD_OUT_DIRECT。

Note that contiguous pages in the virtual address space are almost never contiguous in the physical address space. Hopefully your device is designed to handle that.

请注意，虚拟地址空间中的连续页面在物理地址空间中几乎从不连续。希望您的设备旨在处理该问题。

3. In case I'll have to write the driver myself, how do I do this translation? which functions do I use? Is it mmGetPhysicalAddress()? How do I use it?

3. 如果我必须自己编写驱动程序，我该如何翻译？我使用哪些功能？是 mmGetPhysicalAddress() 吗？我如何使用它？

There's a lot more to writing a driver than just calling a few APIs. If you're going to write a driver, I would recommend reading as much relevant material as you can from MSDNand OSR. Also, look at the examples in the Windows Driver Kit.

编写驱动程序不仅仅是调用一些 API。如果您要编写驱动程序，我建议您从MSDN和OSR 中阅读尽可能多的相关材料。另外，请查看Windows Driver Kit中的示例。

4. Also, if I understand correctly, mmGetPhysicalAddress() returns the physical address of a virtual base address that is in the context of the calling process. But if the calling process is the driver, and I'm using my application to call the driver for that function, I'm changing contexts and I am no longer in the context of the app when the mmGetPhysicalAddress routine is called... so how do I translate the virtual address in the application (user-mode) memory space, not the driver?

4. 另外，如果我理解正确，mmGetPhysicalAddress() 返回调用进程上下文中的虚拟基地址的物理地址。但是，如果调用进程是驱动程序，并且我正在使用我的应用程序调用该函数的驱动程序，那么我正在更改上下文，并且当 mmGetPhysicalAddress 例程被调用时，我不再处于应用程序的上下文中......所以如何转换应用程序（用户模式）内存空间中的虚拟地址，而不是驱动程序？

Drivers are not processes. A driver can run in the context of any process, as well as various elevated contexts (interrupt handlers and DPCs).

驱动程序不是进程。驱动程序可以在任何进程的上下文以及各种提升的上下文（中断处理程序和 DPC）中运行。

回答by Christopher

1) No

1) 否

2) Yes, you have to write a driver. Best would be either a virtual driver, or change the driver for the special-external device.

2）是的，你必须写一个驱动程序。最好是虚拟驱动程序，或者更改特殊外部设备的驱动程序。

3) This gets very confusing here. MmGetPhysicalAddressshould be the method you are looking for, but I really don't know how the physical address is mapped to the bank/chip/etc. on the physical memory.

3）这在这里变得非常混乱。MmGetPhysicalAddress应该是你正在寻找的方法，但我真的不知道物理地址是如何映射到银行/芯片/等的。在物理内存上。

4) You cannot use paged memory, because that gets relocated. You can lock paged memory with MmProbeAndLockPageson an MDL you can build on memory passed in from the user mode calling context. But it is better to allocate non-paged memory and hand that to your user mode application.

4）您不能使用分页内存，因为它会被重新定位。您可以MmProbeAndLockPages在 MDL 上锁定分页内存，您可以在从用户模式调用上下文传入的内存上构建。但是最好分配非分页内存并将其交给您的用户模式应用程序。

PVOID p = ExAllocatePoolWithTag( NonPagedPool, POOL_TAG );
PHYSICAL_ADDRESS realAddr = MmGetPhysicalAddress( p );

// use realAddr

回答by Jake Oshins

You have a virtually continguous buffer in your application. That range of virtual memory is, as you noted, only available in the context of your application and some of it may be paged out at any time. So, in order to do access the memory from a device (which is to say, do DMA) you need to both lock it down and get a description that can be passed to a device.

您的应用程序中有一个几乎连续的缓冲区。正如您所指出的，该虚拟内存范围仅在您的应用程序上下文中可用，并且其中一些可能随时被调出。因此，为了从设备访问内存（也就是说，进行 DMA），您需要锁定它并获得可以传递给设备的描述。

You can get a description of the buffer called an MDL, or Memory Descriptor List, by sending an IOCTL (via the DeviceControl function) to your driver using METHOD_IN_DIRECT or METHOD_OUT_DIRECT. See the following page for a discussion of defining IOCTLs.

您可以通过使用 METHOD_IN_DIRECT 或 METHOD_OUT_DIRECT 向驱动程序发送 IOCTL（通过 DeviceControl 函数）来获取称为 MDL 或内存描述符列表的缓冲区的描述。有关定义 IOCTL 的讨论，请参阅下一页。

http://msdn.microsoft.com/en-us/library/ms795909.aspx

http://msdn.microsoft.com/en-us/library/ms795909.aspx

Now that you have a description of the buffer in a driver for your device, you can lock it down so that the buffer remains in memory for the entire period that your device may act on it. Look up MmProbeAndLockPages on MSDN.

既然您已经在设备的驱动程序中对缓冲区进行了描述，那么您可以将其锁定，以便缓冲区在您的设备可能对其进行操作的整个时间段内都保留在内存中。在 MSDN 上查找 MmProbeAndLockPages。

Your device may or may not be able to read or write all of the memory in the buffer. The device may only support 32-bit DMA and the machine may have more than 4GB of RAM. Or you may be dealing with a machine that has an IOMMU, a GART or some other address translation technology. To accomodate this, use the various DMA APIs to get a set of logical addresses that are good for use by your device. In many cases, these logical addresses will be equivalent to the physical addresses that your question orginally asked about, but not always.

您的设备可能无法读取或写入缓冲区中的所有内存。设备可能只支持 32 位 DMA，机器可能有超过 4GB 的 RAM。或者您可能正在处理具有 IOMMU、GART 或其他一些地址转换技术的机器。为了适应这一点，请使用各种 DMA API 来获取一组适合您的设备使用的逻辑地址。在许多情况下，这些逻辑地址将等同于您的问题最初询问的物理地址，但并非总是如此。

Which DMA API you use depends on whether your device can handle scatter/gather lists and such. Your driver, in its setup code, will call IoGetDmaAdapter and use some of the functions returned by it.

您使用哪个 DMA API 取决于您的设备是否可以处理分散/收集列表等。您的驱动程序在其设置代码中将调用 IoGetDmaAdapter 并使用它返回的一些函数。

Typically, you'll be interested in GetScatterGatherList and PutScatterGatherList. You supply a function (ExecutionRoutine) which actually programs your hardware to do the transfer.

通常，您会对 GetScatterGatherList 和 PutScatterGatherList 感兴趣。您提供一个函数 (ExecutionRoutine)，它实际上对您的硬件进行编程以进行传输。

There's a lot of details involved. Good Luck.

涉及的细节很多。祝你好运。

回答by RandomNickName42

You can not access the page tables from user space, they are mapped in the kernel.

您不能从用户空间访问页表，它们在内核中被映射。

If you are in the kernel, you can simply inspect the value of CR3 to locate the base page table address and then begin your resolution.

如果你在内核中，你可以简单地检查 CR3 的值来定位基页表地址，然后开始你的解析。

This blog serieshas a wonderful explanation of how to do this. You do not need any OS facility/API to resolve virtual<->physical addresses.

这个博客系列很好地解释了如何做到这一点。您不需要任何操作系统设施/API 来解析虚拟<->物理地址。

Virtual Address: f9a10054

1: kd> .formats 0xf9a10054
Binary:  11111001 10100001 00000000 01010100

Page Directory Pointer Index(PDPI)       11                        Index into

1st table(Page Directory Pointer Table) Page Directory Index(PDI)
111001 101 Index into 2nd table(Page Directory Table) Page Table Index(PTI)
00001 0000 Index into 3rd table(Page Table) Byte Index
0000 01010100 0x054, the offset into the physical memory page

虚拟地址：f9a10054

1: kd> .formats 0xf9a10054
Binary:  11111001 10100001 00000000 01010100

Page Directory Pointer Index(PDPI)       11                        Index into

1st table(Page Directory Pointer Table) Page Directory Index(PDI)
111001 101 Index into 2nd table(Page Directory Table) Page Table Index(PTI)
00001 0000 Index into 3rd table(Page Table) Byte Index
0000 01010100 0x054，偏移量物理内存页

In his example, they use windbg, !dq is a physical memory read.

在他的示例中，他们使用windbg，!dq 是物理内存读取。

回答by Ana Betts

You reallyshouldn't be doing stuff like this in usermode; as Christopher says, you need to lock the pages so that mm doesn't decide to page out your backing memory while a device is using it, which would end up corrupting random memory pages.

你真的不应该在用户模式下做这样的事情；正如克里斯托弗所说，您需要锁定页面，以便 mm 不会在设备正在使用它时决定调出您的后备内存，这最终会破坏随机内存页面。

But if the calling process is the driver, and I'm using my application to call the driver for that function, I'm changing contexts and I am no longer in the context of the app when the mmGetPhysicalAddress routine is called

但是，如果调用进程是驱动程序，并且我正在使用我的应用程序调用该函数的驱动程序，我正在更改上下文，并且当 mmGetPhysicalAddress 例程被调用时，我不再处于应用程序的上下文中

Drivers don't have context like user-mode apps do; if you're calling into a driver via an IOCTL or something, you are usually (but not guaranteed!) to be in the calling user thread's context. But really, this doesn't matter for what you're asking, because kernel-mode memory (anything above 0x80000000) is the same mapping no matter where you are, and you'd end up allocating memory in the kernel side. But again, write a proper driver. Use WDF (http://www.microsoft.com/whdc/driver/wdf/default.mspx), and it will make writing a correct driver much easier (though still pretty tricky, Windows driver writing is not easy)

驱动程序不像用户模式应用程序那样具有上下文；如果您通过 IOCTL 或其他方式调用驱动程序，您通常（但不能保证！）处于调用用户线程的上下文中。但实际上，这与您的要求无关，因为无论您身在何处，内核模式内存（0x80000000 以上的任何内容）都是相同的映射，并且最终会在内核端分配内存。但同样，编写一个合适的驱动程序。使用 WDF ( http://www.microsoft.com/whdc/driver/wdf/default.mspx)，它将使编写正确的驱动程序变得容易得多（尽管仍然很棘手，但编写 Windows 驱动程序并不容易）

EDIT: Just thought I'd throw out a few book references to help you out, you should definitely (even if you don't pursue writing the driver) read Windows Internals by Russinovich and Solomon (http://www.amazon.com/Microsoft-Windows-Internals-4th-Server/dp/0735619174/ref=pd_bbs_sr_2?ie=UTF8&s=books&qid=1229284688&sr=8-2); Programming the Microsoft Windows Driver Model is good too (http://www.amazon.com/Programming-Microsoft-Windows-Driver-Second/dp/0735618038/ref=sr_1_1?ie=UTF8&s=books&qid=1229284726&sr=1-1)

编辑：只是想我会抛出一些参考书来帮助你，你绝对应该（即使你不追求编写驱动程序）阅读 Russinovich 和 Solomon 的 Windows Internals ( http://www.amazon.com /Microsoft-Windows-Internals-4th-Server/dp/0735619174/ref=pd_bbs_sr_2?ie=UTF8&s=books&qid=1229284688&sr=8-2); 对 Microsoft Windows 驱动程序模型进行编程也很好（http://www.amazon.com/Programming-Microsoft-Windows-Driver-Second/dp/0735618038/ref=sr_1_1?ie=UTF8&s=books&qid=1229284726&sr=1-1）

回答by jyoung

Wait, there is more. For the privilege of runnning on your customer's Vista 64 bit, you get expend more time and money to get your kernal mode driver resigned my Microsoft,

等等，还有更多。为了获得在您客户的 Vista 64 位上运行的特权，您将花费更多的时间和金钱来让您的内核模式驱动程序辞去我的微软职务，