MS with MultiLoginApp - Access UI elements, focus browser

[Warren]

I'm using Multiloginapp which launches it's own custom version of Chrome and Firefox browsers with a variety of built in tools to prevent browser fingerprinting.

I'm not sure whether I can use the Web Recorder tools at all since:

1) It requires opening in it's own browser which then means I'm opening the profiles WITHOUT the spoofing tools thus defeating the purpose of using MLA

2) Web recorder appears to only support IE, so not sure if any of what it records could be transferred over to Chrome or FF playback.

3) Even if some of what it captured could be transferred to stock Chrome or FF, still not sure it would work for those launched with MLA since none of the tools (other than image recognition) in MS appear to see anything inside of browser windows launched with MLA. Find Object Wizard, for instance, sees only the forward, back, refresh, and address bar. Everything in the window itself is undifferentiated unless a modal pops up (even some of those aren't recognized, but perhaps they aren't true modals)

I would more than love to be corrected on that point if there is some way to detect, select, or otherwise interact with anything inside the browser window.

If not, though, and it's back to image recognition, some basic things are tripping me up. One example is just setting focus to the browser window since it keeps changing names depending on which tab is open and what page is loaded on that tab. This is causing problems for me trying to do basic things like just setting the focus on the browser window.

I tried using handles, but they're just numbers, and the numbers keep changing as well.

So, trying to see if there's a way to focus according to parent or something like that. If I use Find Element Wizard over page, I get:

Window: 5C0294 [Chrome_RenderWidgetHostHWND] Top level: 2C038C [Chrome_WidgetWin_1]

That last bit of Chrome_WidgetWin_1 is the bit that never seems to change (at least when using Chrome rather than Firefox)

Is there a way of focusing on that since it is constant?

Also love to hear if I am misinterpreting inability to get UI element info within the Multiloginapp browser windows.

thx

[Marcus Tettmar]

WebRecorder wraps IE and all the IE functions work against IE, hence the name. They allow you to access the DOM within IE. Chrome doesn't unfortunately expose that interface. So for Chrome you would need to use methods you would use to automate desktop apps. E.g. image recognition, key/mouse sends, and also try the UIAccessible functions (UIClick, UIFocus etc).

Use View System Windows to find the Chrome tab and then right click to see accessible list. You will see all the html element names and values. Using these you can easily UIClick on a link in the page. E.g.:

UIClick>Macro Scheduler Forums*,Board index

First though you need to enable the accessibility features in Chrome. To do this go to:

chrome://accessibility

[Warren]

OK

Unfortunately, user enabled chrome settings don't persist across launches with MLA (at least not the few I've tried so far). I'll contact the devs to see if there is a workaround, or see about setting the UI accessibility as a first step upon launch.

The image recognition is actually working quite well (especially when tolerance is dialed in), but I'm trying to stay a step ahead of site changes wherever possible, and eventually to create fallback procedures in case a step fails.

Will experiment. thx

[Marcus Tettmar]

If you look at that chrome://accessibility page it says on the right that you need to start Chrome with command line switches to persist those settings. It tells you what switches are needed. So you would just need to modify your user's shortcuts accordingly.