C# SSH.NET SFTP 递归获取目录和文件列表

SSH.NET SFTP Get a list of directories and files recursively

提问by user1462119

I am using Renci.SshNet library to get a list of files and directories recursively by using SFTP. I can able to connect SFTP site but I am not sure how to get a list of directories and files recursively in C#. I haven't found any useful examples.

我正在使用 Renci.SshNet 库通过使用 SFTP 递归获取文件和目录列表。我可以连接 SFTP 站点，但我不确定如何在 C# 中递归获取目录和文件列表。我还没有找到任何有用的例子。

Has anybody tried this? If so, can you post some sample code about how to get these files and folders recursively.

有人试过这个吗？如果是这样，您能否发布一些有关如何递归获取这些文件和文件夹的示例代码。

Thanks,
Prav

谢谢，
普拉夫

回答by shane

Try this:

尝试这个：

var filePaths = client.ListDirectory(client.WorkingDirectory);

回答by Carlo Bos

This library has some quirks that make this recursive listing tricky because the interaction between the ChangeDirectoryand ListDirectorydo not work as you may expect.

该库有一些怪癖，使这个递归上市棘手，因为之间的相互作用ChangeDirectory和ListDirectory不工作，你可能期望。

The following does notlist the files in the /home directory instead it lists the files in the / (root) directory:

以下并没有列出在/ home目录中的文件，而不是它列出了/（根）目录下的文件：

sftp.ChangeDirectory("home");
sftp.ListDirectory("").Select (s => s.FullName);

The following does notwork and returns a SftpPathNotFoundException:

下面确实没有工作，并返回一个SftpPathNotFoundException：

sftp.ChangeDirectory("home");
sftp.ListDirectory("home").Select (s => s.FullName);

The following is the correct way to list the files in the /home directory

以下是列出/home目录下文件的正确方法

sftp.ChangeDirectory("/");
sftp.ListDirectory("home").Select (s => s.FullName);

This is pretty crazy if you ask me. Setting the default directory with the ChangeDirectorymethod has no effect on the ListDirectorymethod unless you specify a folder in the parameter of this method. Seems like a bug should be written for this.

如果你问我，这很疯狂。除非您在此方法的参数中指定文件夹，否则使用该ChangeDirectory方法设置默认目录不会影响ListDirectory该方法。似乎应该为此编写一个错误。

So when you write your recursive function you'll have to set the default directory once and then change the directory in the ListDirectorycall as you iterate over the folders. The listing returns an enumerable of SftpFiles. These can then be checked individually for IsDirectory == true. Just be aware that the listing also returns the .and ..entries (which are directories). You'll want to skip these if you want to avoid an infinite loop. :-)

因此，当您编写递归函数时，您必须设置一次默认目录，然后在ListDirectory遍历文件夹时更改调用中的目录。该列表返回一个可枚举的 SftpFiles。然后可以单独检查这些IsDirectory == true。请注意，该列表还会返回.和..条目（它们是目录）。如果您想避免无限循环，您需要跳过这些。:-)

EDIT 2/23/2018

编辑 2/23/2018

I was reviewing some of my old answers and would like to apologize for the answer above and supply the following working code. Note that this example does not require ChangeDirectory, since it's using the Fullnamefor the ListDirectory:

我正在我的一些旧答案，并想为上述答案道歉并提供以下工作代码。请注意，此示例不需要ChangeDirectory，因为它使用的是Fullnamefor ListDirectory：

void Main()
{
    using (var client = new Renci.SshNet.SftpClient("sftp.host.com", "user", "password"))
    {
        var files = new List<String>();
        client.Connect();
        ListDirectory(client, ".", ref files);
        client.Disconnect();
        files.Dump();
    }
}

void ListDirectory(SftpClient client, String dirName, ref List<String> files)
{
    foreach (var entry in client.ListDirectory(dirName))
    {

        if (entry.IsDirectory)
        {
            ListDirectory(client, entry.FullName, ref files);
        }
        else
        {
            files.Add(entry.FullName);
        }
    }
}

回答by Flemin Adambukulam

I have achieved this using recursion. Created a class TransportResponse like this

我已经使用递归实现了这一点。像这样创建了一个类 TransportResponse

 public class TransportResponse
{
    public string directoryName { get; set; }
    public string fileName { get; set; }
    public DateTime fileTimeStamp { get; set; }
    public MemoryStream fileStream { get; set; }
    public List<TransportResponse> lstTransportResponse { get; set; }
}

I create a list of TransportResponse class. If the directoryName is not null, it will contain a list of the same class which will have the the files inside that directory as a MemoryStream ( this can be changed as per your use case)

我创建了一个 TransportResponse 类的列表。如果 directoryName 不为空，它将包含一个相同类的列表，该列表将该目录中的文件作为 MemoryStream （这可以根据您的用例进行更改）

List<TransportResponse> lstResponse = new List<TransportResponse>();
using (var client = new SftpClient(connectionInfo))
  {
          try
          {
                    Console.WriteLine("Connecting to " + connectionInfo.Host + " ...");
                    client.Connect();
                    Console.WriteLine("Connected to " + connectionInfo.Host + " ...");
           }
           catch (Exception ex)
           {
                    Console.WriteLine("Could not connect to "+ connectionInfo.Host +" server. Exception Details: " + ex.Message);
           }
           if (client.IsConnected)
           {
                    var files = client.ListDirectory(transport.SourceFolder);
                    lstResponse = downloadFilesInDirectory(files, client);
                    client.Disconnect();
            }
            else
            {
                    Console.WriteLine("Could not download files from "+ transport.TransportIdentifier +" because client was not connected.");
             }
   }



private static List<TransportResponse> downloadFilesInDirectory(IEnumerable<SftpFile> files, SftpClient client)
    {
        List<TransportResponse> lstResponse = new List<TransportResponse>();
        foreach (var file in files)
        {
            if (!file.IsDirectory)
            {
                if (file.Name != "." && file.Name != "..")
                {
                    if (!TransportDAL.checkFileExists(file.Name, file.LastWriteTime))
                    {
                        using (MemoryStream fs = new MemoryStream())
                        {
                            try
                            {
                                Console.WriteLine("Reading " + file.Name + "...");
                                client.DownloadFile(file.FullName, fs);
                                fs.Seek(0, SeekOrigin.Begin);
                                lstResponse.Add(new TransportResponse { fileName = file.Name, fileTimeStamp = file.LastWriteTime, fileStream = new MemoryStream(fs.GetBuffer()) });
                            }
                            catch(Exception ex)
                            {
                                Console.WriteLine("Error reading File. Exception Details: " + ex.Message);
                            }
                        }
                    }
                    else
                    {
                        Console.WriteLine("File was downloaded previously");
                    }
                }
            }
            else
            {
                if (file.Name != "." && file.Name != "..")
                {
                    lstResponse.Add(new TransportResponse { directoryName = file.Name,lstTransportResponse = downloadFilesInDirectory(client.ListDirectory(file.Name), client) });
                }                
            }
        }

        return lstResponse;
    }

Hope this helps. Thanks

希望这可以帮助。谢谢

回答by Pedro Vicente

@Carlos Bos

@卡洛斯博斯

This library has some quirks that make this recursive listing tricky because the interaction between the ChangeDirectory and ListDirectory do not work as you may expect.

这个库有一些怪癖，使这个递归列表变得棘手，因为 ChangeDirectory 和 ListDirectory 之间的交互不像你期望的那样工作。

correct

正确的

It works well when the ChangeDirectory() parameter is "."

当 ChangeDirectory() 参数为“.”时，它运行良好。

but if you do

但如果你这样做

SftpClient sftp ...;
sftp.ChangeDirectory("some_folder");
//get file list
List<SftpFile> fileList = sftp.ListDirectory("some_folder").ToList();

then there is an assertion because the ListDirectory() call expects "some_folder/some_folder"

然后有一个断言，因为 ListDirectory() 调用需要“some_folder/some_folder”

The workaround I use is to save and restore the current directory before a remote upload/rename to "some_folder", and you need to list that folder before the operation (e.g to see the file already exists)

我使用的解决方法是在远程上传/重命名为“some_folder”之前保存和恢复当前目录，并且您需要在操作之前列出该文件夹（例如查看文件已经存在）

string working_directory = sftp.WorkingDirectory;
sftp.ChangeDirectory("some_folder");
sftp.RenameFile("name", "new_name");
sftp.ChangeDirectory(working_directory);

to check if the file exists, this call is sufficient

检查文件是否存在，这个调用就足够了

sftp.Exists(path)

or if you want to add some other criteria, like case sensitive or not

或者如果您想添加一些其他条件，例如是否区分大小写

 public FileExistence checkFileExists(string folder, string fileName)
    {
      //get file list
      List<SftpFile> fileList = sftp.ListDirectory(folder).ToList();

      if (fileList == null)
      {
        return FileExistence.UNCONFIRMED;
      }

      foreach (SftpFile f in fileList)
      {
        Console.WriteLine(f.ToString());
        //a not case sensitive comparison is made
        if (f.IsRegularFile && f.Name.ToLower() == fileName.ToLower())
        {
          return FileExistence.EXISTS;
        }
      }

      //if not found in traversal , it does not exist
      return FileExistence.DOES_NOT_EXIST;
    }

where FileExistence is

FileExistence 在哪里

public enum FileExistence
    {
      EXISTS,
      DOES_NOT_EXIST,
      UNCONFIRMED
    };

回答by jbusciglio acuity

Here is a full class. It's .NET Core 2.1 Http trigger function app (v2)

这是一个完整的类。它是 .NET Core 2.1 Http 触发函数应用程序 (v2)

I wanted to get rid of any directories that start with '.', cause my sftp server has .cache folders and .ssh folders with keys. Also didn't want to have to deal with folder names like '.' or '..'

我想删除所有以“.”开头的目录，因为我的 sftp 服务器有 .cache 文件夹和带有密钥的 .ssh 文件夹。也不想处理像“.”这样的文件夹名称。或者 '..'

What I will end up doing is projecting the SftpFile into a type that I work with and return that to the caller (in this case it will be a logic app). I'll then pass that object into a stored procedure and use OPENJSON to build up my monitoring table. This is basically the first step in creating my SFTP processing queue that will move files off my SFTP folder and into my Data Lake (blob for now until I come up with something better I guess).

我最终要做的是将 SftpFile 投影到我使用的类型并将其返回给调用者（在这种情况下它将是一个逻辑应用程序）。然后我将该对象传递到一个存储过程并使用 OPENJSON 来建立我的监控表。这基本上是创建我的 SFTP 处理队列的第一步，它将把文件从我的 SFTP 文件夹移到我的数据湖（现在是 blob，直到我想出更好的东西为止）。

The reason I used .WorkingDirectory is because I created a user with home directory as '/home'. This lets me traverse all of my user folders. My app doesn't need to have a specific folder as a starting point, just the user 'root' so to speak.

我使用 .WorkingDirectory 的原因是因为我创建了一个主目录为“/home”的用户。这让我可以遍历我的所有用户文件夹。我的应用程序不需要有一个特定的文件夹作为起点，可以这么说只是用户“root”。

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
using Renci.SshNet;
using Renci.SshNet.Sftp;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace SFTPFileMonitor
{
    public class GetListOfFiles
    {
        [FunctionName("GetListOfFiles")]
        public async Task<IActionResult> RunAsync([HttpTrigger(AuthorizationLevel.Anonymous, "get")] HttpRequest req, ILogger log)
        {
            log.LogInformation("C# HTTP trigger function processed a request.");

            List<SftpFile> zFiles;
            int fileCount;
            decimal totalSizeGB;
            long totalSizeBytes;

            using (SftpClient sftpClient = new SftpClient("hostname", "username", "password"))
            {
                sftpClient.Connect();
                zFiles = await GetFiles(sftpClient, sftpClient.WorkingDirectory, new List<SftpFile>());
                fileCount = zFiles.Count;
                totalSizeBytes = zFiles.Sum(l => l.Length);
                totalSizeGB = BytesToGB(totalSizeBytes);
            }

            return new OkObjectResult(new { fileCount, totalSizeBytes, totalSizeGB, zFiles });
        }
        private async Task<List<SftpFile>> GetFiles(SftpClient sftpClient, string directory, List<SftpFile> files)
        {
            foreach (SftpFile sftpFile in sftpClient.ListDirectory(directory))
            {
                if (sftpFile.Name.StartsWith('.')) { continue; }

                if (sftpFile.IsDirectory)
                {
                    await GetFiles(sftpClient, sftpFile.FullName, files);
                }
                else
                {
                    files.Add(sftpFile);
                }
            }
            return files;
        }
        private decimal BytesToGB(long bytes)
        {
            return Convert.ToDecimal(bytes) / 1024 / 1024 / 1024;
        }
    }
}