windows fopen()、fclose() 究竟是如何工作的？

How exactly does fopen(), fclose() work?

提问by Fabian

I was just wondering about the functions fopen, fclose, socket and closesocket. When calling fopen or opening a socket, what exactly is happening (especially memory wise)?

我只是想知道 fopen、fclose、socket 和 closesocket 函数。调用 fopen 或打开套接字时，究竟发生了什么（尤其是内存方面）？

Can opening files/sockets without closing them cause memory leaks?

打开文件/套接字而不关闭它们会导致内存泄漏吗？

And third, how are sockets created and what do they look like memory wise?

第三，套接字是如何创建的，它们在内存方面看起来如何？

I'm also interrested in the role of the operating system (Windows) in reading the sockets and sending the data.

我还对操作系统 (Windows) 在读取套接字和发送数据方面的作用感兴趣。

回答by Stefan Monov

Disclaimer: I'm mostly unqualified to talk about this. It'd be great if someone more knowledgeable posted too.

免责声明：我基本上没有资格谈论这个。如果有知识渊博的人也发帖就好了。

Files

文件

The details of how things like fopen() are implemented will depend a lot on the operating system (UNIX has fopen() too, for example). Even versions of Windows can differ a lot from each other.

如何实现 fopen() 之类的细节在很大程度上取决于操作系统（例如，UNIX 也有 fopen()）。甚至 Windows 的版本也可能有很大不同。

I'll give you my idea of how it works, but it's basically speculation.

我会告诉你它是如何工作的，但这基本上是猜测。

• When called, fopen allocates a FILEobject on the heap. Note that the data in a FILE object is undocumented - FILE is an opaque struct, you can only use pointers-to-FILE from your code.
• The FILE object gets initialized. For example, something like fillLevel = 0where fillLevel is the amount of buffered data that hasn't been flushed yet.
• A call to the filesystem driver (FS driver) opens the file and provides a handle to it, which is put somewhere in the FILE struct.
  • To do this, the FS driver figures out the HDD address corresponding to the requested path, and internally remembers this HDD address, so it can later fulfill calls to fread etc.
    • The FS driver uses a sort of indexing table (stored on the HDD) to figure out the HDD address corresponding to the requested path. This will differ a lot depending on the filesystem type - FAT32, NTFS and so on.
    • The FS driver relies on the HDD driver to perform the actual reads and writes to the HDD.
• A cache might be allocated in RAM for the file. This way, if the user requests 1 byte to be read, C++ may read a KB just in case, so later reads will be instantaneous.
• A pointer to the allocated FILE gets returned from fopen.

• 调用时，fopen在堆上分配一个FILE对象。请注意， FILE 对象中的数据是未记录的 - FILE 是一个不透明的 struct，您只能在代码中使用指向 FILE 的指针。
• FILE 对象被初始化。例如，fillLevel = 0fillLevel 是尚未刷新的缓冲数据量。
• 对文件系统驱动程序（FS 驱动程序）的调用会打开文件并为其提供句柄，该句柄位于 FILE 结构中的某处。
  • 为此，FS 驱动程序计算出与请求的路径对应的硬盘地址，并在内部记住该硬盘地址，以便稍后执行对 fread 等的调用。
    • FS 驱动程序使用一种索引表（存储在 HDD 上）来计算与请求路径对应的 HDD 地址。这将有很大不同，具体取决于文件系统类型 - FAT32、NTFS 等。
    • FS 驱动程序依赖于 HDD 驱动程序来执行对 HDD 的实际读取和写入。
• 可能会在 RAM 中为文件分配缓存。这样，如果用户请求读取 1 个字节，C++ 可能会读取 KB 以防万一，因此后面的读取将是即时的。
• 从 fopen 返回指向已分配 FILE 的指针。

If you open a file and never close it, some things will leak, yes. The FILE struct will leak, the FS driver's internal data will leak, the cache (if any) will leak too.

如果你打开一个文件并且从不关闭它，有些东西会泄漏，是的。FILE 结构将泄漏，FS 驱动程序的内部数据将泄漏，缓存（如果有）也会泄漏。

But memory is not the only thing that will leak. The file itselfwill leak, because the OS will think it's open when it's not. This can become a problem for example in Windows, where a file opened in write-mode cannot be opened in write-mode again until it's been closed.

但内存并不是唯一会泄漏的东西。该文件本身会泄漏，因为操作系统会认为它是开放的，当它不是。这可能会成为一个问题，例如在 Windows 中，以写入模式打开的文件在关闭之前无法再次以写入模式打开。

If your app exits without closing some file, most OSes will clean up after it. But that's not much use, because your app will probably run for a long time before exiting, and during that time, it will still need to properly close all files. Also, you can't fully rely on the OS to clean up after you - it's not guaranteed in the C Standard.

如果您的应用程序在没有关闭某些文件的情况下退出，大多数操作系统都会在它之后进行清理。但这并没有多大用处，因为您的应用程序可能会在退出之前运行很长时间，并且在此期间，它仍然需要正确关闭所有文件。此外，您不能完全依赖操作系统在您之后进行清理 - 在 C 标准中不能保证。

Sockets

插座

A socket's implementation will depend on the type of socket - network listen socket, network client socket, inter-process socket, etc.

套接字的实现取决于套接字的类型——网络监听套接字、网络客户端套接字、进程间套接字等。

A full discussion of all types of sockets and their possible implementations wouldn't fit here.

此处不适合对所有类型的套接字及其可能的实现进行全面讨论。

In short:

简而言之：

• just like a file, a socket keeps some info in RAM, describing things relevant to its operation, such as the IP of the remote host.
• it can also have caches in RAM for performance reasons
• it can hold onto finite OS resources such as open ports, making them unavailable for use by other apps

• 就像文件一样，套接字在 RAM 中保存一些信息，描述与其操作相关的内容，例如远程主机的 IP。
• 出于性能原因，它还可以在 RAM 中缓存
• 它可以保留有限的操作系统资源，例如开放端口，使其无法供其他应用程序使用

All these things will leak if you don't close the socket.

如果您不关闭套接字，所有这些东西都会泄漏。

The role of the OS in sockets

操作系统在套接字中的作用

The OS implements the TCP/IP standard, Ethernet and other protocols needed to schedule/dispatch/accept connections and to make them available to user code via an API like Berkeley Sockets.

操作系统实现了 TCP/IP 标准、以太网和其他协议，这些协议需要调度/分派/接受连接，并通过像 Berkeley Sockets 这样的 API 将它们提供给用户代码。

The OS will delegate network I/O (communication with the network card) to the network driver.

操作系统会将网络 I/O（与网卡的通信）委托给网络驱动程序。

回答by jw_

With VS2017 on Windows 10, you can see the internal by callstack:

使用Windows 10上的VS2017，可以通过callstack查看内部：

ntdll.dll!NtCreateFile()   Unknown
KernelBase.dll!CreateFileInternal() Unknown
KernelBase.dll!CreateFileW()   Unknown
ucrtbased.dll!create_file(const wchar_t * const path, _SECURITY_ATTRIBUTES * const security_attributes, const `anonymous-namespace'::file_options options) Line 234 C++
ucrtbased.dll!_wsopen_nolock(int * punlock_flag, int * pfh, const wchar_t * path, int oflag, int shflag, int pmode, int secure) Line 702    C++
ucrtbased.dll!_sopen_nolock(int * punlock_flag, int * pfh, const char * path, int oflag, int shflag, int pmode, int secure) Line 852    C++
ucrtbased.dll!__crt_char_traits<char>::tsopen_nolock<int * __ptr64,int * __ptr64,char const * __ptr64 const & __ptr64,int const & __ptr64,int,int const & __ptr64,int>(int * && <args_0>, int * && <args_1>, const char * const & <args_2>, const int & <args_3>, int && <args_4>, const int & <args_5>, int && <args_6>) Line 109  C++
ucrtbased.dll!common_sopen_dispatch<char>(const char * const path, const int oflag, const int shflag, const int pmode, int * const pfh, const int secure) Line 172  C++
ucrtbased.dll!_sopen_dispatch(const char * path, int oflag, int shflag, int pmode, int * pfh, int secure) Line 204  C++
ucrtbased.dll!_sopen_s(int * pfh, const char * path, int oflag, int shflag, int pmode) Line 895 C++
ucrtbased.dll!__crt_char_traits<char>::tsopen_s<int * __ptr64,char const * __ptr64 const & __ptr64,int const & __ptr64,int const & __ptr64,int>(int * && <args_0>, const char * const & <args_1>, const int & <args_2>, const int & <args_3>, int && <args_4>) Line 109 C++
ucrtbased.dll!common_openfile<char>(const char * const file_name, const char * const mode, const int share_flag, const __crt_stdio_stream stream) Line 38   C++
ucrtbased.dll!_openfile(const char * file_name, const char * mode, int share_flag, _iobuf * public_stream) Line 67  C++
ucrtbased.dll!__crt_char_traits<char>::open_file<char const * __ptr64 const & __ptr64,char const * __ptr64 const & __ptr64,int const & __ptr64,_iobuf * __ptr64>(const char * const & <args_0>, const char * const & <args_1>, const int & <args_2>, _iobuf * && <args_3>) Line 109 C++
ucrtbased.dll!common_fsopen<char>(const char * const file_name, const char * const mode, const int share_flag) Line 54  C++
ucrtbased.dll!fopen(const char * file, const char * mode) Line 104  C++

Most code are in:

大多数代码在：

C:\Program Files (x86)\Windows Kits\Source.0.17763.0\ucrt\stdio\fopen.cpp
C:\Program Files (x86)\Windows Kits\Source.0.17763.0\ucrt\stdio\openfile.cpp
C:\Program Files (x86)\Windows Kits\Source.0.17763.0\ucrt\lowio\open.cpp

In _wsopen_nolock in open.cpp, there is:

在open.cpp的_wsopen_nolock中，有：

// Allocate the CRT file handle.  Note that if a handle is allocated, it is
// locked when it is returned by the allocation function.  It is our caller's
// responsibility to unlock the file handle (we do not unlock it before
// returning).
*pfh = _alloc_osfhnd();

Finally, it calls Windows API CreateFileW which calls hiden API "NtCreateFile" whose assembly code is:

最后，它调用 Windows API CreateFileW，后者调用隐藏 API“NtCreateFile”，其汇编代码为：

NtCreateFile:
00007FFFD81A0120 mov         r10,rcx  
00007FFFD81A0123 mov         eax,55h  
00007FFFD81A0128 test        byte ptr[7FFE0308h],1  
00007FFFD81A0130 jne         NtCreateFile+15h(07FFFD81A0135h)
00007FFFD81A0132 syscall
00007FFFD81A0134 ret
00007FFFD81A0135 int         2Eh  
00007FFFD81A0137 ret
00007FFFD81A0138 nop         dword ptr[rax + rax]

So finally it execute the syscall instruction which goes into kernel code.

所以最后它执行进入内核代码的系统调用指令。