wpf 将键/值对列表转换为数据表

Convert a list of key/value pairs to datatable

提问by chersun

I'm working on a parser. It gets values from source text. It does not know beforehand how many or which values it will get, i.e. names of variables, their count etc. could vary greatly. Each section of source provides some values only, not a complete list. Those values are currently stored in a list of custom class, similar to KeyValuePair, but written from scratch.

我正在研究解析器。它从源文本中获取值。它事先不知道它将获得多少或哪些值，即变量的名称、它们的计数等可能会有很大差异。源的每个部分仅提供一些值，而不是完整列表。这些值当前存储在自定义类的列表中，类似于 KeyValuePair，但从头开始编写。

Sample what is retrieved from source:

从源中检索的示例：

Section 1:
    KeyA = ValA1
    KeyB = ValB1
    KeyD = ValD1
Section 2:
    KeyC = ValC2
Section 3:
    KeyB = ValB3
    KeyD = ValD3

etc.

等等。

Now, I'd like to show this information to user as a DataGrid in form of:

现在，我想以以下形式将这些信息作为 DataGrid 显示给用户：

| KeyA  | KeyB  | KeyC  | KeyD  |
+-------+-------+-------+-------+
| ValA1 | ValB1 |       | ValD1 |
|       |       | ValC2 |       |
|       | ValB3 |       | ValD3 |

Currently, I'm iterating through all values found in each section, check if column exists - if not - creating new column. If column exists - adding value to respective row/column. Then attaching resulting DataTable to DataGrid as:

目前，我正在遍历每个部分中找到的所有值，检查列是否存在 - 如果不存在 - 创建新列。如果列存在 - 向相应的行/列添加值。然后将生成的 DataTable 附加到 DataGrid 为：

dg.ItemSource=dt.AsDataView();

This works perfectly as intended, yet, that is too slow.

这完全按预期工作，但是太慢了。

I'd appreciate any thoughts on how I could speed that up. Either initial storing, or convertion to DataTable, or some other way of binding data to achieve the same presentation to user.

我很感激有关如何加快速度的任何想法。初始存储或转换为 DataTable，或其他绑定数据的方式以实现对用户的相同呈现。

C#, WPF, .NET framework 4.5

C#、WPF、.NET 框架 4.5

Update:All loading and processing is done beforehand. Ready data is stored as a tree of processed sections. Each section as one of properties holds a list of key/value pairs. Each section has class to populate given DataTable with it's values.

更新：所有加载和处理都是事先完成的。就绪数据存储为已处理部分的树。作为属性之一的每个部分都包含一个键/值对列表。每个部分都有用它的值填充给定 DataTable 的类。

I.e. data on backend looks like:

即后端的数据看起来像：

File1
  + Section 1 on level 1
  |   + Section 1
  |   + Section 2
  + Section 2 on level 1
  + Section 3 on level 1
  |   + Section 1
  |   + Section 2
  |   + Section 3
  |   + Section 4
  + Section 4
File2 ...

Each Section has a method:

每个Section都有一个方法：

public void CollectValues(DataTable target) {...}

Which is called by higher level element with some DataTable (initially - empty and getting filled as it goes).

它由具有一些 DataTable 的更高级别元素调用（最初 - 空并且随着它的进行而被填充）。

Each section contains internal variable:

每个部分都包含内部变量：

private List<CustomValue> Values;

Which holds all the already found&processed values in CustomValue class. CustomValue ~= KeyValuePair, but with added processing routines.

它保存了 CustomValue 类中所有已经找到和处理的值。CustomValue ~= KeyValuePair，但添加了处理例程。

So what happens is CollectValues is being called from requested level (could be top, could be any other) with empty unprepared DataTable. CollectValues iterates (foreach) through all available values in list on current level and adds them to target DataTable 1 at a time, prior to that checking if DataColumn exists with needed name (target[Value.Key]!=null) - and creating column before attempting to add respective value if needed. In metacode:

所以会发生什么是从请求的级别（可能是最高的，也可能是任何其他级别）调用 CollectValues，并使用空的未准备好的 DataTable。CollectValues 迭代（foreach）当前级别列表中的所有可用值，并将它们一次添加到目标 DataTable 1，然后检查是否存在具有所需名称的 DataColumn (target[Value.Key]!=null) - 并创建列如果需要，在尝试添加相应的值之前。在元代码中：

public void CollectValues(DataTable target)
{
    DataRow dr = target.Rows.Create();
    foreach(var pair in Values)
    {
        if(target[pair.Key]==null) target.Columns.Add(...);
        dr[pair.Key] = pair.Value;
    }
    foreach(var child in Children)
        child.CollectValues(target);
}

Why this specific part - values is just part of similar routines. Other routines crawl similarly on same data set, retrieving other things (mostly working with lists, no DataTables) - all of them work near instantly. Collecting DataTable though might take a few seconds for 1 source for resulting DataGrid to get populated.

为什么这个特定部分 - 值只是类似例程的一部分。其他例程在同一数据集上类似地爬行，检索其他内容（主要使用列表，没有 DataTables）——所有这些都几乎立即工作。虽然收集 DataTable 可能需要几秒钟才能为 1 个源填充生成的 DataGrid。

Average amount of Values rarely exceeds 1000 (like, 10 columns by 100 rows). DataTable is attached to DataGrid only after it was fully populated.

值的平均数量很少超过 1000（例如，10 列乘 100 行）。DataTable 仅在完全填充后才附加到 DataGrid。

Just for info on sizes: Sources - usually 2 to 10 files. Each source text size can range 100Kb - 100 MB. Usual file size is around 1-2 MB. Size of backend data in memory usually is under 100 MB.

仅用于大小信息：来源 - 通常为 2 到 10 个文件。每个源文本的大小范围为 100Kb - 100 MB。通常的文件大小约为 1-2 MB。内存中后端数据的大小通常在 100 MB 以下。

And to highlight again. It's only DataTable that worries me. Highlights, Sectioning, source retrieval, filtering etc. - all works within my expectations. So I'm looking first of all - for a way to optimize conversion from list of key/value pairs to DataTable, or for a way to store those values differently initially (after processing) to speed up process.

并再次强调。只有数据表让我担心。亮点、切片、源检索、过滤等 - 一切都在我的预期之内。所以我首先在寻找一种方法来优化从键/值对列表到 DataTable 的转换，或者一种方法来最初（处理后）以不同的方式存储这些值以加快过程。

Hope this gives enough info. Not listing source currently to reduce size.

希望这能提供足够的信息。目前不列出源以减少大小。

回答by Bobson

I'd look for a data structure other than a DataTable to use here. It sounds to me like what you need is a Dictionary<string, Dictionary<int, CustomValue>>. The stringis your column name, the intis an ID for the row of data, and CustomValueis the data itself.

我会在此处寻找 DataTable 以外的数据结构。在我看来，您需要的是Dictionary<string, Dictionary<int, CustomValue>>. 该string是你的列名，int是为数据行的ID，并且CustomValue是数据本身。

public void CollectValues(Dictionary<string, Dictionary<int, CustomValue>> target)
{
    foreach(var pair in Values)
    {
        if(target[pair.Key]==null) target.Add(new Dictionary<int, CustomValue>());
        target[pair.Key].Add(pair.ID, pair.Value);
    }
    foreach(var child in Children)
        child.CollectValues(target);
}

If you don't already have an pair.IDin place, you can just use a counter variable (either staticor passed with each call) so that each object has a different ID.

如果您还没有pair.ID就位，您可以只使用一个计数器变量（static或者通过每次调用传递），以便每个对象都有不同的 ID。

---

---

It might make more sense to store the values by row, with the columns that each set of data has, rather than the reverse. That would be a IEnumerable<Dictionary<string, CustomValue>>, with each Dictionaryrepresenting one row. You would pull out all the columns with target.Select(x => x.Key).Distinct().

按行存储值可能更有意义，每组数据具有列，而不是相反。那将是 a IEnumerable<Dictionary<string, CustomValue>>，每个Dictionary代表一行。你会用target.Select(x => x.Key).Distinct().

回答by paparazzo

DataTable is slow. It does a lot of stuff.

数据表很慢。它做了很多事情。

If you are all string then I would create a collection

如果你们都是字符串，那么我会创建一个集合

List<String> ColNames;
List<String> ColValues;

List<ColValues> RowsColValues;

Then you need to manually bind the columns to the DataGrid using ColValues[i] syntax.

然后您需要使用 ColValues[i] 语法手动将列绑定到 DataGrid。

And for speed use ListView GridView for this.
DataGrid is slow and bulkly compared to Gridview.
But GridView does not edit.

为了速度，为此使用 ListView GridView。
与 Gridview 相比，DataGrid 速度慢且体积庞大。
但 GridView 不编辑。

Not making this up.
I do exactly this but a different scenario.
User selects the columns they want to see.

不是编的。
我正是这样做的，但情况不同。
用户选择他们想要查看的列。

DyamicColumns

动态柱