java 创建 Flash 游戏机器人

Creating a flash game bot

提问by Neutralise

Does anyone have any recommendations for resources or methods on how to create a bot in java which can play a flash game?

有没有人对如何在java中创建可以玩Flash游戏的机器人的资源或方法有任何建议？

I am thinking of using the Robot class to watch the screen and make actions, but I need ways of finding images in images, etc. I am sure this has been done before but google searches return alot of nonsense..

我正在考虑使用 Robot 类来观看屏幕并执行操作，但我需要在图像中查找图像的方法等。我确定以前已经这样做过，但谷歌搜索返回了很多废话..

回答by Lex

I have just built a casual card playing bot (not poker) in about 3 days and it was overwhelmingly easy. The method below will work for most button based games. Of course for 3D games and fast moving 2D graphics you will need a different approach.

我刚刚在大约 3 天内构建了一个休闲纸牌游戏机器人（不是扑克），这非常容易。下面的方法适用于大多数基于按钮的游戏。当然，对于 3D 游戏和快速移动的 2D 图形，您将需要不同的方法。

It's strange that there is so little useful information out there. All the articles on writing bots will link to Windows stuff with DLLs, hooks, memory snooping and all kinds of other tricks. And of course, you dont need any of this nonsense. All you need is Java and awt.Robot.

奇怪的是，有用的信息很少。所有关于编写机器人的文章都将链接到带有 DLL、钩子、内存窥探和各种其他技巧的 Windows 内容。当然，你不需要任何这些废话。您只需要 Java 和 awt.Robot。

Emitting clicks is straightforward enough, the game logic is something you have to figure out on your own, so I will focus on getting the screen Input.

发出点击非常简单，游戏逻辑是你必须自己弄清楚的，所以我将专注于获取屏幕输入。

You start by getting a fullscreen capture. Then you find a distinct image that is present in every frame, and use that image as an anchor to calculate the offset of the top left corner of the game interface. The anchor image could be anything, even a two or three pixel wide border between the game and the browser background, just make sure to make it long enough. Then you simply search the screen capture to find where this image is located, this is a very simple algorithm that compares images one pixel at the time and changes the offset until the match is found. Once you have the UI offset, you find the relative offset to all the buttons on the screen, and take a little snapshot for each button you want to interact with. You use these offsets to detect active buttons or other indicators. As a bonus you can also use these offsets when clicking on buttons. You should make a collection of png (not jpg) images of all the buttons you want to detect. Then you run simple image comparison algorithms against the content of the screen capture.

您首先获得全屏捕获。然后你会发现每一帧中都有一个独特的图像，并使用该图像作为锚点来计算游戏界面左上角的偏移量。锚点图像可以是任何东西，甚至是游戏和浏览器背景之间的两个或三个像素宽的边框，只要确保它足够长即可。然后您只需搜索屏幕截图以找到该图像的位置，这是一种非常简单的算法，它比较图像一个像素的时间并更改偏移量，直到找到匹配项。获得 UI 偏移量后，您将找到屏幕上所有按钮的相对偏移量，并为每个要与之交互的按钮拍摄一个小快照。您可以使用这些偏移量来检测活动按钮或其他指示器。作为奖励，您还可以在单​​击按钮时使用这些偏移量。您应该收集要检测的所有按钮的 png（而非 jpg）图像。然后针对屏幕截图的内容运行简单的图像比较算法。

Flash does vector font rendering and does not produce pixel-perfect consistency between different instances of the game. Also image scaling will not be consistent. Here are the ways to fight this:

Flash 进行矢量字体渲染，并且不会在游戏的不同实例之间产生像素完美的一致性。图像缩放也不会一致。以下是对抗这种情况的方法：

1. Filtering. You can utilize many filtering methods, the simplest one is a threshold filter. Simply make all the pixels 100% black if the intensity is less that your threshold, or 100% white otherwise. Then run pixel-by-pixel comparison on the filtered images and find the number of distinct pixels. Assume a match for low values of distinct pixels. You may also have to shift the images by + or - 1 pixel in each direction, for a total of 3*3 = 9 comparisons. Just choose the lowest distinct pixel value as a result.
2. Average color. If you are simply looking if an indicator (for example a green glowing circle) is present on the screen or not, just find the average color in the rectangle where the indicator is and see if it's close enough to the average color of the stored image.

1. 过滤。您可以使用多种过滤方法，最简单的一种是阈值过滤器。如果强度小于阈值，只需将所有像素设为 100% 黑色，否则设为 100% 白色。然后对过滤后的图像进行逐像素比较，找出不同像素的数量。假设匹配不同像素的低值。您可能还需要在每个方向上将图像移动 + 或 - 1 个像素，总共进行 3*3 = 9 次比较。只需选择最低的不同像素值即可。
2. 平均颜色。如果您只是查看屏幕上是否存在指示符（例如绿色发光圆圈），只需在指示符所在的矩形中找到平均颜色，然后查看它是否足够接近存储图像的平均颜色.

Font recognition can be tricky. If you can separate the letters (usually with monochrome fonts), then you can simply use the Filtering method above to recognize each letter individually.

字体识别可能很棘手。如果您可以将字母分开（通常使用单色字体），那么您可以简单地使用上面的过滤方法来单独识别每个字母。

If you hit a wall, you can try a variety of more complicated filtering methods (blur + intensity threshold is a good combo).

如果碰壁，可以尝试各种更复杂的过滤方法（模糊+强度阈值是一个很好的组合）。

The final tips are:

最后的提示是：

• Keep your image recognition and logic completely separate.
• Invest in writing good debug tools that dump the information on the image detection process so that you can tweak the filters and threshold values until they work. This debug-dump should include the filtered images being compared, so you get a good feel on how the filters work, what went wrong, and how to fix it. You will not succeed without these tools, so if you are planning to skip them, then don't bother at all.
• Have a working prototype early. Even if it only detects one button.
• Do not optimize any image algorithms until you have a working bot.

• 将您的图像识别和逻辑完全分开。
• 投资编写好的调试工具，转储图像检测过程的信息，以便您可以调整过滤器和阈值，直到它们工作。这个调试转储应该包括被比较的过滤图像，这样你就可以很好地了解过滤器是如何工作的，出了什么问题，以及如何修复它。如果没有这些工具，您将不会成功，因此如果您打算跳过它们，请不要打扰。
• 尽早有一个工作原型。即使它只检测到一个按钮。
• 在你有一个可以工作的机器人之前，不要优化任何图像算法。

回答by Garrett Hall

Making an intelligent bot is difficult. If you can get away with just using timing and scripted clicks, do it. Otherwise try to make decisions based on grabbing pixels - this will be fragile(can break on different browsers/screens/game updates), but easier to implement than a more robust image processing method. Keep your logic and screen scraping code separate so you can test the logic independently and update the scraping code without affecting the logic.

制作智能机器人很困难。如果您可以通过使用计时和脚本点击来逃脱，那就去做吧。否则尝试根据抓取像素做出决定 - 这将是脆弱的（可能会在不同的浏览器/屏幕/游戏更新上中断），但比更强大的图像处理方法更容易实现。将您的逻辑和屏幕抓取代码分开，以便您可以独立测试逻辑并在不影响逻辑的情况下更新抓取代码。

I also recommend looking into AutoIt. With it you can pretty easily script repetitive actions, as well as grab text and pixel data off the screen. I've used it to script Elements, Dream World, and several other games. Tell me the game you are trying to bot and I could help you more.

我还建议查看AutoIt。有了它，您可以非常轻松地编写重复操作的脚本，以及从屏幕上抓取文本和像素数据。我用它来编写 Elements、Dream World 和其他几个游戏的脚本。告诉我你正在尝试机器人的游戏，我可以为你提供更多帮助。

回答by h3xStream

You could modify the flash application and provide a simple api to the browser (Javascript and possibly Java applet). The api would contains methods for controls and to access the state of the game. ExternalInterface.addCallbackwill make this possible.

您可以修改 flash 应用程序并向浏览器提供一个简单的 api（Javascript 和可能的 Java 小程序）。api 将包含用于控制和访问游戏状态的方法。ExternalInterface.addCallback将使这成为可能。

回答by Ryan

There are easier ways to do this. If you insist on using Java to game bot, you will need to use external libraries in order to both read what is going on in the game and to provide input to the computer.

有更简单的方法可以做到这一点。如果您坚持使用 Java 来玩游戏机器人，您将需要使用外部库来读取游戏中发生的事情并向计算机提供输入。

In addition Java is generally bloated and there are a lot of OOP features that are not needed for a very simple program. I agree with Garrett Hall's suggestion of using AutoIt to read the game and provide input to the computer. There is more to that than game botting of course, but AutoIt can do most of what you need to write a flash game bot.

此外，Java 通常是臃肿的，并且有很多 OOP 特性对于一个非常简单的程序来说是不需要的。我同意 Garrett Hall 的建议，即使用 AutoIt 读取游戏并向计算机提供输入。当然，除了游戏机器人之外还有更多内容，但 AutoIt 可以完成编写 Flash 游戏机器人所需的大部分工作。