Dealing with dynamic window names

[Warren]

I've mentioned this in passing before, but it's becoming a pervasive issue, so I want to nail it down once and for all. Basically, situation is the following:

Each machine has multiple chrome (custom chromium) windows open simultaneously, and each of those has multiple active tabs. Every piece of this puzzle seems to be dynamic. If I click on the window frames with object wizard I get something like:

Window: 54020C [Chrome_WidgetWin_1]

If I click within the frame on the website within that same frame, I get something like:

Window: 580478 [Chrome_RenderWidgetHostHWND] Top level: 54020C [Chrome_WidgetWin_1]

If I click on another tab in same window, I get another like the above where the top level matches, but other window number deoesn't.

Straight forward enough... except that it never matches from machine to machine, and even if I relaunch the same exact profile and hit the same exact sites on the same exact tabs, ALL the numbers change. Top level, interior windows... all different. I've played around with trying to use handles, and I just can't find persistent info to work with.

So... I'd think I'd want to use window NAMES like "Chrome", etc. Unfortunately, the window names change constantly depending on which tab I'm on, and even which subpage of a site I happen to be on within that page... often without a single persistent word to use a wildcard with. Even when I can, it only works when there's a single browser window open as multiple sessions open plus wildcard names often leads to MS selecting the wrong window.

I've done my best for days now to work this out, but I'm just not seeing how to consistently do simple things like SetFocus. I'm getting the impression I need to be finding parents, then parsing which child window or something like that... but again, the numbers are different every time, and on every machine.

Just looking for the simplest way to say things like set focus on the Chrome session 1, and switch to tab 2. I've been handling the tabs via chrome keyboard shortcuts. Am I going about this all wrong?

[Marcus Tettmar]

It's quite hard to offer solutions without seeing things in more detail/in the flesh. It's very rare that we don't know some part of the window title. You can use Regular Expressions in window titles if substring matching isn't good enough. But if you really can't use the window title you can use FindObject, using the class name and index, and parent. You can nest these if you need to drill down.

You can't use handles without first identifying the object with class name. Handles are assigned at runtime when the object is created, so will change from session to session or even more often - e.g. when a tab is opened it will be created and assigned a handle by the system. What won't change is the class name. Usually you know the creation order. To automate something you would have to know the sequence of events and therefore you should be able to predict z order.

[Marcus Tettmar]

It's quite hard to offer solutions without seeing things in more detail/in the flesh. It's very rare that we don't know some part of the window title. You can use Regular Expressions in window titles if substring matching isn't good enough. But if you really can't use the window title you can use FindObject, using the class name and index, and parent. You can nest these if you need to drill down.

You can't use handles without first identifying the object with class name. Handles are assigned at runtime when the object is created, so will change from session to session or even more often - e.g. when a tab is opened it will be created and assigned a handle by the system. What won't change is the class name. Usually you know the creation order. To automate something you would have to know the sequence of events and therefore you should be able to predict z order.

[zabros2020]

Hi Warren,

Here is something i've come up with. Hope it helps with part of your solution
To test it, open a few browser windows with multiple tabs for each one.

Code: Select all

'Get the browser and tab handles and names
GoSub>GetBrowserTabs

'to test, loop through all.
Let>i=0
Repeat>i
  Add>i,1
  Let>browserHandle=arrBrowserHandle_%i%
  'each tab
  Let>iTab=0
  Repeat>iTab
    Add>iTab,1
    Let>numTab=arrNumBrowserTab_%i%
    Let>tabName=arrBrowserTab_%i%_%iTab%
    'focus on browser window
    SetFocus>%browserHandle%
    'select the tab
    UIClick>%browserHandle%,%tabName%
    MessageModal>browser:%SPACE%%SPACE%%SPACE%%i%%CRLF%%CRLF%browser tab_:%SPACE%%SPACE%%SPACE%%tabName%
  Until>iTab=%numTab%
Until>i=%iBrowser%

SRT>GetBrowserTabs
  '---------------------------------------------------------------------
  'Gets list of browser windows matching browserName var
  'Assigns handler for window in array
  'Gets tab title related to window by parsing result of UIAccessibleList>
  'Uses
  '----------------------------------------------------------------------
  Let>WIN_USEHANDLE=0
  Let>browserName=Google Chrome
  GetWindowList>wlist
  Separate>wlist,%CRLF%,arrWindowList
  Let>i=0
  Let>iBrowser=0
  Let>FoundPDF=0
  Repeat>i
    Add>i,1
    Let>sText=arrWindowList_%i%
    'look for browser name
    Position>%browserName%,%sText%,1,iPos
    If>iPos>0
      'get the handle for each browserName window, then each tab window name
      GetWindowHandle>%sText%,hWndParent
      'browser window handle array
      Add>iBrowser,1
      ArrayDim>arrBrowserHandle,%iBrowser%
      Let>arrBrowserHandle_%iBrowser%=%hWndParent%
      'force win handle
      Let>WIN_USEHANDLE=1
      UIAccessibleList>%hWndParent%,res
      Separate>res,=,arrTabList
      Let>iList=0
      Repeat>iList
        Add>iList,1
        Let>sText=arrTabList_%iList%
        Let>lookFor={"New Tab"}
        Let>lookUntil=main
        Position>%lookFor%,%sText%,1,iPos
        If>iPos>0
          Let>iInnList=%iList%
          Let>iTabList=0
          Repeat>iInnList
            Add>iInnList,1
            Let>sText1=arrTabList_%iInnList%
            Position>%lookUntil%,%sText1%,1,iPos
            If>iPos>0
              'exit all inner loops, we have found all tabs
              Let>iList=%arrTabList_Count%
              Let>iInnList=%arrTabList_Count%

            Else

              Position>Close,%sText1%,1,iPos
              If>iPos=0
                'get the tab name(element name)
                Position>%CRLF%,%sText1%,1,posStart
                Length>%sText1%,chkLen
                MidStr>%sText1%,%posStart%,%chkLen%,sText1
                Trim>%sText1%,sText1
                'build array tab list
                Add>iTabList,1
                Let>arrBrowserTab_%iBrowser%_%iTabList%=%sText1%
                Let>arrNumBrowserTab_%iBrowser%=%iTabList%
                'skip duplicate
                Add>iInnList,1
              EndIf
            EndIf
          Until>iInnList=%arrTabList_Count%
        EndIf
      Until>iList=%arrTabList_Count%
    EndIf
  Until>i=%arrWindowList_Count%

END>GetBrowserTabs

[Warren]

Hi Warren,

Here is something i've come up with. Hope it helps with part of your solution
To test it, open a few browser windows with multiple tabs for each one.

That definitely works for identifying the tabs and keeping the different windows distinct. It's a lot to go through, so still parsing some of it to figure out how to integrate.

I have noticed that it can reassign which is browser 1 or 2 depending on if another browser has been launched since. That will definitely be an issue... especially since the different windows will often contain tabs identical to those in other windows.

Since the profiles (MLA term for distinct browser identities that persist across sessions... equivalent to browser windows for purposes of this discussion) all have names that are used to launch them from the MLA app, it makes most sense to use those names, so that I'm tracking "ProfileA" as opposed to "browser1". The trick is then figuring out how to "stick" that name to that browser window so the tabs in ProfileA don't start reporting as belonging to ProfileB once ProfileB is launched.

It's a lot to process given that I'm trying to do what would seem to be a relatively straightforward task. Though the names may change, I always know which tab I want to manipulate, and which profile it's on, so for all the commands that need a handle or window name or whatever, all I'm really trying to do with any of this is to find a reliable way in a dynamic naming environment to say:

Do something on Tab1 in ProfileA

I can see a path here to dig deeper into the script you've provided (thanks for that), but can't help but wonder if I'm overcomplicating things somehow.

[zabros2020]

I see.

It does sound like a simply task, but I can't seem to find a simple process for it. Unless Marcus builds us a new function that will get the info.

I have refactored code to fix the issue you mentioned "I have noticed that it can reassign which is browser 1 or 2 depending on if another browser has been launched since. That will definitely be an issue... especially since the different windows will often contain tabs identical to those in other windows. "

Hope it helps

Code: Select all

'Get the browser and tab handles and names
GoSub>GetBrowserTabs

'to test, loop through all.
Let>i=0
Repeat>i
  Add>i,1
  Let>browserHandle=arrBrowserHandle_%i%
  'each tab
  Let>iTab=0
  Repeat>iTab
    Add>iTab,1
    Let>numTab=arrNumBrowserTab_%i%
    Let>tabName=arrBrowserTab_%i%_%iTab%
    'focus on browser window
    SetFocus>%browserHandle%
    'select the tab
    UIClick>%browserHandle%,%tabName%
    MessageModal>browser:%SPACE%%SPACE%%SPACE%%i%%CRLF%%CRLF%browser tab_:%SPACE%%SPACE%%SPACE%%tabName%
  Until>iTab=%numTab%
Until>i=%iBrowser%

MessageModal>Create a new "browsertab"
'run every time you need to check for new tabs or windows
'the first order of handles remains the same throughout
GoSub>GetBrowserTabs

'to test, loop through all again,
'difference this time is we use same sequence or browser handles in array
'as it was captured in first run
Let>i=0
Repeat>i
  Add>i,1
  Let>browserHandle=arrBrowserHandle_%i%
  'each tab
  Let>iTab=0
  Repeat>iTab
    Add>iTab,1
    Let>numTab=arrNumBrowserTab_%i%
    Let>tabName=arrBrowserTab_%i%_%iTab%
    'focus on browser window
    SetFocus>%browserHandle%
    'select the tab
    UIClick>%browserHandle%,%tabName%
    MessageModal>browser:%SPACE%%SPACE%%SPACE%%i%%CRLF%%CRLF%browser tab_:%SPACE%%SPACE%%SPACE%%tabName%
  Until>iTab=%numTab%
Until>i=%iBrowser%

SRT>GetBrowserTabs
  '---------------------------------------------------------------------
  'Gets list of browser windows matching browserName var
  'Assigns handler for window in array
  'Gets tab title related to window by parsing result of UIAccessibleList>
  'Uses
  '----------------------------------------------------------------------
  Let>WIN_USEHANDLE=0
  'Google Chrome
  Let>browserName=Google Chrome
  'Mozilla Firefox
  'Let>browserName=Mozilla Firefox
  GetWindowList>wlist
  Separate>wlist,%CRLF%,arrWindowList
  Let>i=0
  Let>iBrowser=0
  Let>FoundPDF=0
  Repeat>i
    Add>i,1
    Let>sText=arrWindowList_%i%
    'look for browser name
    Position>%browserName%,%sText%,1,iPos
    If>iPos>0
      'get the handle for each browserName window, then each tab window name
      GetWindowHandle>%sText%,hWndParent
      'browser window handle array
      Add>iBrowser,1
      '***new code***
      '----------------------------------------------------------
      'if array element already assigned, we know we already have
      'this hansle. no need to recapture, use it instead.
      'thiss allows for the browser sequence and order to remain.
      'if a new tab is added, this is capture as usual
      '-----------------------------------------------------------
      Assigned>arrBrowserHandle_%iBrowser%,isAssigned
      If>isAssigned=TRUE
        Let>hWndParent=arrBrowserHandle_%iBrowser%
      Else
        ArrayDim>arrBrowserHandle,%iBrowser%
        Let>arrBrowserHandle_%iBrowser%=%hWndParent%
      EndIf
      'force win handle
      Let>WIN_USEHANDLE=1

      UIAccessibleList>%hWndParent%,res
      Separate>res,=,arrTabList
      Let>iList=0
      Repeat>iList
        Add>iList,1
        Let>sText=arrTabList_%iList%
        Let>lookFor={"New Tab"}
        Let>lookUntil=main
        Position>%lookFor%,%sText%,1,iPos
        If>iPos>0
          Let>iInnList=%iList%
          Let>iTabList=0
          Repeat>iInnList
            Add>iInnList,1
            Let>sText1=arrTabList_%iInnList%
            Position>%lookUntil%,%sText1%,1,iPos
            If>iPos>0
              'exit all inner loops, we have found all tabs
              Let>iList=%arrTabList_Count%
              Let>iInnList=%arrTabList_Count%

            Else

              Position>Close,%sText1%,1,iPos
              If>iPos=0
                'get the tab name(element name)
                Position>%CRLF%,%sText1%,1,posStart
                Length>%sText1%,chkLen
                MidStr>%sText1%,%posStart%,%chkLen%,sText1
                Trim>%sText1%,sText1
                'build array tab list
                Add>iTabList,1
                Let>arrBrowserTab_%iBrowser%_%iTabList%=%sText1%
                Let>arrNumBrowserTab_%iBrowser%=%iTabList%
                'skip duplicate
                Add>iInnList,1
              EndIf
            EndIf
          Until>iInnList=%arrTabList_Count%
        EndIf
      Until>iList=%arrTabList_Count%
    EndIf
  Until>i=%arrWindowList_Count%

END>GetBrowserTabs

[Warren]

OK, thx.

I've been digging into it a bit, and gets the initial read right, then gets confused once anything changes.

If I launch a browser window with 5 tabs, it will select and name each tab, and state that each belongs to Browser 1. So far, so good, but if I then when prompted to create a new browser tab, if I add another tab to that same browser window (default home page Google.com), it declares the new Google tab first, then (in order, each is a separate modal):

browser tab_: Menu containing hidden bookmarks
browser tab_: Managed Bookmarks

Then, Google page switches to chrome://apps, and I get:

browser tab_: Apps
browser tab_: Seperator
browser tab_: For quick access, place your bookmarks here on the bookmarks bar.

Then, it keeps the chrome://apps page, but creates another new tab (default homepage Google.com), jumps to it, and says:

browser tab_: New Tab

Then cycles through all the actual tabs again before quitting.


If, on the other hand, I launch a second browser window when modal says to create a new browser tab, it goes through all the tabs on browser 1 (calling it browser 1 again), but then it goes throgh all the same tabs again on browser 1, but calls it "browser 2, so it has now swtiched the name of the browser window, and nver acknowledges the tabs on the actual browser 2.

If I launch the entire script again, it sometimes gets it right up to the point where it asks me to create a new browser tab), and sometimes does as previously desribed where it goes through all the B1 tabs as B1, then goes through them again as B2.