java 如何读取窗口内容（使用accessibilityService）并在Android中使用draw over其他应用程序权限来调用UI？

How to read window content (using accessibilityService) and evoking UI using draw over other app permission in Android?

提问by Umesh Chauhan

My lastquestion about the same topic was not clear enough and was put on hold by community and later it was automatically deleted. So, I am explaining that Question in detailed manner so that community can understand and help in a better manner.

我上一个关于同一主题的问题不够清楚，被社区搁置，后来被自动删​​除。所以，我正在详细解释这个问题，以便社区可以更好地理解和帮助。

I want functionality similar to Voodoo Appand MySmartPriceoffers.

我想要类似于Voodoo App和MySmartPrice提供的功能。

Now what they do

现在他们做什么

Step 1:When we open the Voodoo App for the first time they show a small tutorial. At the end of tutorial there is a "Activate Now" button which when pressed take us to Accessibility settings screen.

第 1 步：当我们第一次打开 Voodoo 应用程序时，他们会显示一个小教程。在教程结束时，有一个“立即激活”按钮，按下后会将我们带到辅助功能设置屏幕。

Step 2:In Accessibility Screen it further guides how to find and unable the Voodoo Service.

第 2 步：在辅助功能屏幕中，它进一步指导如何查找和禁用 Voodoo 服务。

Step 3:When we enable it, it further ask to grant "Observe Your Action" and "Retrieve Window Content" permissions.

第 3 步：当我们启用它时，它会进一步要求授予“观察您的操作”和“检索窗口内容”权限。

Step 4:Once we are done granting permissions on accessibility screen, and move to some shopping app or access shopping site via browser and reach product page a voodoo floating button pops up automatically.

第 4 步：一旦我们在可访问性屏幕上授予权限，并通过浏览器移动到某个购物应用程序或访问购物网站并到达产品页面，就会自动弹出一个巫毒浮动按钮。

Step 5:Once we click on it, it shows the price of same/related/similar products available on other apps or website so that user can check for best deal available.

第 5 步：一旦我们点击它，它就会显示其他应用程序或网站上可用的相同/相关/类似产品的价格，以便用户可以查看最优惠的价格。

Reference Screenshot of Voodoo

Voodoo 参考截图

Now what I want to know form community:

现在我想从社区了解什么：

1. How I can show my help UI over Accessibility screen and product detail page.
2. How to detect when a product page is there on screen and to evoke my floating button.
3. How to fetch name of product shown on screen.
4. How to restrict this screen reading functionality for some apps only? (as I don't want to end up in some copyright issue)
5. Is there any tutorial which can help me out? Although I had already tried on Google for any direct tutorial and didn't got any success.

1. 我如何在辅助功能屏幕和产品详细信息页面上显示我的帮助 UI。
2. 如何检测产品页面何时出现在屏幕上并唤起我的浮动按钮。
3. 如何获取屏幕上显示的产品名称。
4. 如何仅针对某些应用程序限制此屏幕阅读功能？（因为我不想以版权问题告终）
5. 有什么教程可以帮助我吗？虽然我已经在 Google 上尝试过任何直接教程，但没有取得任何成功。

Now why I need this information:

现在为什么我需要这些信息：

I am planning a app for students which will help them to get there desired (or similar) book (ebook) for free if it is available on my server or on some web location which is available web. I want to restrict it the functionality to some apps only due to copyright issues on some books.

我正在为学生计划一个应用程序，如果它在我的服务器上或某个可用的网络位置上可用，它将帮助他们免费获得所需的（或类似的）书籍（电子书）。由于某些书籍的版权问题，我想将其功能限制为某些应用程序。

Now what I know from my study regarding this topic (but I am not sure whether I am on right track or not)

现在我从我对这个主题的研究中了解到的（但我不确定我是否在正确的轨道上）

• I can take help form Accessibility Service
• Draw Over Other Apps. Example 1, Example 2

• 我可以从无障碍服务中获取帮助
• 绘制其他应用程序。例一、例二

回答by Umesh Chauhan

I had tried to summarize how i was able to get thru the problem in a blog post. Anyone who needs some help can take a look into it.

我曾尝试在博客文章中总结我如何解决问题。任何需要帮助的人都可以查看它。

Accessibility Service to the next level

无障碍服务更上一层楼

My Previous answer was deleted by moderators may due to i just posted a link to blog. So i am posting my answer again with details here too.

我之前的回答被版主删除了，可能是因为我刚刚发布了一个博客链接。所以我也在这里再次发布我的答案并提供详细信息。

The easiest way is provided by android itself. If you are building something on your own app this is the best and easiest possible way of doing it. You have to use “findAccessibilityNodeInfosByViewId ()” as explained below

最简单的方法是android本身提供的。如果您正在自己的应用程序上构建某些东西，这是最好和最简单的方法。您必须使用“findAccessibilityNodeInfosByViewId ()”，如下所述

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
    AccessibilityNodeInfo source = event.getSource(); 
    if (source == null) {
        return; 
    } 
    List<AccessibilityNodeInfo> findAccessibilityNodeInfosByViewId = source.findAccessibilityNodeInfosByViewId("YOUR PACKAGE NAME:id/RESOURCE ID FROM WHERE YOU WANT DATA"); 
    if (findAccessibilityNodeInfosByViewId.size() > 0) {
        AccessibilityNodeInfo parent = (AccessibilityNodeInfo) findAccessibilityNodeInfosByViewId.get(0);
        // You can also traverse the list if required data is deep in view hierarchy. 
        String requiredText = parent.getText().toString();
        Log.i("Required Text", requiredText);
    }
}  

If you are building something on other apps and don't know the res id. You have to build a logic with the combination of parent, child count, level on which your data is found, type of event. You have to look for a pattern in event or source or the combination of above said values to get the required data preciously. Further, you have to setup some tweak in your code to get the accurate data according to your task. Like in our task we have some regex to get precious data.

如果您正在其他应用程序上构建某些内容并且不知道 res id。您必须使用父级、子级计数、数据所在级别、事件类型的组合来构建逻辑。您必须在事件或来源或上述值的组合中寻找模式，才能获得所需的宝贵数据。此外，您必须在代码中设置一些调整以根据您的任务获得准确的数据。就像在我们的任务中，我们有一些正则表达式来获取宝贵的数据。

You have to take care that if the view or UI is changed or res ids are changed your current logic might fail. So, you have to keep a close look at the app on which you are building your service. Following are some example code in which we are getting data without res id that might be useful for you to understand how it will work.

您必须注意，如果视图或 UI 更改或 res id 更改，您当前的逻辑可能会失败。因此，您必须密切关注正在构建服务的应用程序。以下是一些示例代码，其中我们在没有 res id 的情况下获取数据，这可能有助于您了解它的工作原理。

To print source and event

打印源和事件

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
    AccessibilityNodeInfo source = event.getSource();
    if (source == null) {
        return;
    } 
    Log.i("Event", event.toString() + ""); 
    Log.i("Source", source.toString() + "");
}

To get the childcount

获取 childcount

source.getChildCount();

If child count is >0 you might have to look into it.

如果孩子计数> 0，您可能需要查看它。

Example Code 1

示例代码 1

if (level == 0 && source.getClassName().equals("android.widget.TextView") && source.getText()!=null && !source.getText().toString().isEmpty()){
    // here level is iteration of for loop
    String recivedText = source.getText().toString(); 
    if(source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.FrameLayout") && source.getParent().getParent()==null){ 
        return recivedText;
    }
}

Example Code 2

示例代码 2

if (source.getPackageName().equals("PACKAGE NAME OF APP FOR WHICH YOUR EXPECTING EVENT")) {
    if(event.getEventType()==AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED && "COMPLETE NAME OF ACTIVITY CLASS WITH PACKAGE NAME (if you want it for some specific screen)".equals(event.getClassName())){
        if(source.getText()!=null && source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.RelativeLayout")&& source.getParent().getParent()!=null && source.getParent().getParent().getClassName().equals("android.widget.ScrollView") && source.getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getClassName().equals("android.support.v4.view.ViewPager")&& source.getParent().getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getParent().getClassName().equals("android.widget.FrameLayout")&& source.getParent().getParent().getParent().getParent().getParent()==null){
        return source.getText().toString();
        }
}

It is bit clumsy but to get it working you have to look deep into logs to find a pattern which will take you to your required data.

这有点笨拙，但要使其正常工作，您必须深入查看日志以找到一种模式，它将带您找到所需的数据。

Update 20/11/2017Google is suspending all apps using accessibility services for things other than accessibility. I had received a email regarding the same for one of my app and other developers are also receiving same email from google (Reddit). So, I think we should look for some other way of doing things and I request you to please update it here also, if some one comes up with other implementation to achieve same behavior it would help others also.

2017 年 11 月 20 日更新Google 将暂停所有使用辅助功能服务的应用程序，用于非辅助功能。我收到了一封关于我的一个应用程序的电子邮件，其他开发人员也收到了来自谷歌（Reddit）的相同电子邮件。所以，我认为我们应该寻找一些其他的做事方式，我请求你也请在这里更新它，如果有人提出其他实现来实现相同的行为，它也会帮助其他人。

Thanks Umesh Chauhan

感谢 Umesh Chauhan

回答by Umesh Chauhan

I have made successfully this type of application like voodoo.and get Flipkart product page title, price etc successfully. the following method used to get user open product screen.

我已经成功地制作了这种类型的应用程序，如 voodoo.并成功获得了 Flipkart 产品页面标题、价格等。以下方法用于让用户打开产品屏幕。

private boolean isOpenFlipkartProductScreen(AccessibilityNodeInfo nodeInfo){
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR2) {
        if(nodeInfo.getViewIdResourceName()!=null){
            if(nodeInfo.getViewIdResourceName().equals("com.flipkart.android:id/callout_last_linearlayout")){
                Utils.sPrint("You just open product page : " + nodeInfo.getClass());
                isOpenProductPage = true;
                return true;
            }
        }
    }
    return false;
}

and following code to used get product title name and price.

并使用以下代码获取产品名称和价格。

private String getObtainTitle(AccessibilityNodeInfo sourceInfo){

    if(sourceInfo == null)
        return "";

    AccessibilityNodeInfo nodeInfo = sourceInfo.getParent().getParent();
    if(nodeInfo!=null){
        int i = 0;
        int i2 = 0;
        for(int i3=0;i3<nodeInfo.getChildCount();i3++){

            try{
                if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR2) {
                    if(nodeInfo.getChild(i3).getViewIdResourceName().equals("com.flipkart.android:id/title_layout")) {
                        i = 0;
                        //   while ( i < nodeInfo.getChild(i).getChildCount()){
                        AccessibilityNodeInfo child = nodeInfo.getChild(i3).getChild(i);
                        String charSequence = child.getText().toString();
                        //   Utils.sPrint("Title is : " + charSequence);
                        if(charSequence!=null){
                            return  charSequence;
                        }
                        // }

                    }
                }
               /*
               *
               * Below code to used for getProduct prise in flipkart app product
               *
               * */
                if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR2) {
                    if(nodeInfo.getChild(i3).getViewIdResourceName().equals("com.flipkart.android:id/price_layout")){
                        AccessibilityNodeInfo child = nodeInfo.getChild(i3).getChild(i);
                        String charSequence = child.getText().toString();

                        if (child.getText().toString().toUpperCase().startsWith("RS.") ||
                                child.getText().toString().toUpperCase().startsWith("\u20b9")) {
                            String replace = charSequence.replace("Rs.",
                                    "").replace("\u20b9", "");

                            //return replace
                            //   Utils.sPrint("Prise is : " + replace);
                        }

                    }
                }
            }catch (Exception ex){
              //  Utils.sPrint("Ex : " + ex.getMessage());
                return "";
            }

        }
    }
    return "";
}