Chrome Driver - Traverse Iframes & Insert HTML

Ideas for new features & functions

Moderators: Dorian (MJT support), JRL

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Mon Mar 02, 2020 11:22 pm

If would be great if we could have a way of traversing Iframes with Chrome Driver. This is possible with Selenium (which uses ChromeDriver to control Chrome):

Code: Select all

driver.switchTo().frame("//iframe[@src='iframe.html']")
Kantu, a chrome addon that runs a simplified version of Selenium uses the index method of traversing frames, with selectFrame=0 referring to the parent page. Something like that might be a good idea.

I would also like to suggest adding support for execute_script, so that we can inject code into HTML. This is a great feature to be able to fall back on if all attempts at targeting the element fail. eg:

Code: Select all

element = driver.find_element_by_id("#green-widgets")
driver.execute_script("arguments[0].innerHTML = '<button class="static-class-not-dynamic-data-value" onclick="doThis()">Click Here</button>';", element)

User avatar
Marcus Tettmar
Site Admin
Posts: 7251
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by Marcus Tettmar » Tue Mar 03, 2020 9:53 am

We can already switch frames using a custom subroutine which you can see below.

UPDATE: edited to show how to identify frame by attribute

We're aiming to add this as a native function (ChromeSwitchContext?) once we've ironed out a few details and worked out what else everyone wants ;-)

This example goes to the contact page on mjtnet.com which hosts the actual form in a frame, so makes a useful example (but please don't spam our contact page by actually submitting anything!). Note the subroutine called SwitchFrame that I have created which you can drop into your own scripts. It takes two parameters - the session ID and the frame index, element object reference json, or null to switch back to the parent window.

Code: Select all

//Start chrome
Let>CHROMEDRIVER_EXE=c:\drive_d\chromedriver.exe
ChromeStart>sessionID

//navigate to mjt's contact form which uses a frame with the fields in it
ChromeNavigate>sessionID,url,https://www.mjtnet.com/contact.htm

//find the frame using its src attribute
ChromeFindElements>sessionID,xpath,//iframe[@src='/contact-form/formpage.html'],elements

//switch browsing context to that frame
GoSub>SwitchFrame,sessionID, {"ELEMENT":"%elements_1%"}

//you can also do it simply by index, zero being the first frame, avoiding using ChromeFindElements if you know its index
//GoSub>SwitchFrame,sessionID,0

//now anything we do is inside that frame so we should be able to get the name field and enter something
ChromeFindElements>sessionID,id,name,inputs
ChromeSetElementValue>sessionID,inputs_1,marcus

ChromeFindElements>sessionID,id,email,inputs
ChromeSetElementValue>sessionID,inputs_1,[email protected]

//we would want to switch back to parent window at some point otherwise subsequent actions will be in the frame
GoSub>SwitchFrame,sessionID,null

ChromeFindElements>sessionID,xpath,//a[@href='/forum/'],forumlinks
ChromeElementAction>sessionID,forumlinks_2,click
Wait>2
ChromeNavigate>sessionID,url,https://www.google.com/

//takes session ID and frame identifier to switch browsing context to
SRT>SwitchFrame
  Let>HTTP_POSTJSON=1
  Let>body= { "id": %SwitchFrame_Var_2% }
  //if you are not using the standard port and have changed it with CHROMEDRIVER_PORT then change the port number here accordingly
  HTTPRequest>http://localhost:9515/session/%SwitchFrame_Var_1%/frame,,POST,body,theResult
END>SwitchFrame
To "traverse" frames you would just keep switching. Each time you switch you change the browsing context. So if there's a frame inside the frame you just switched to just use SwitchFrame again ...

This subroutine should work for the Edge functions too - just change the port to 17556 or whatever you have set EDGEDRIVER_PORT to.

Enjoy!
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

User avatar
Marcus Tettmar
Site Admin
Posts: 7251
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by Marcus Tettmar » Tue Mar 03, 2020 10:40 am

Updated reply above to show how to identify frames by attributes.

To answer the other point about code injection. Here's another custom subroutine:

Code: Select all

SRT>ExecuteJS
  Let>HTTP_POSTJSON=1
  Let>body= { "script": "%ExecuteJS_Var_2%", "args": [] }
  //if you are not using the standard port and have changed it with CHROMEDRIVER_PORT then change the port number here accordingly
  HTTPRequest>http://localhost:9515/session/%ExecuteJS_Var_1%/execute,,POST,body,theResult
END>ExecuteJS
Call it with .e.g.:

Code: Select all

GoSub>ExecuteJS,sessionID,alert(\"hello\")
So for example after switching frames to the contact form you could set the name field using javascript instead:

Code: Select all

GoSub>ExecuteJS,sessionID,document.getElementById(\"name\").value=\"fred\"
Again, if this is something people find useful we can turn this into a native function.
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

User avatar
Marcus Tettmar
Site Admin
Posts: 7251
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by Marcus Tettmar » Tue Mar 03, 2020 8:59 pm

Well, since we were working on this anyway and it was further ahead than I thought, I managed to reprioritise a few things and get an update pushed out today with added native ChromeSwitchFrame and EdgeSwitchFrame functions. Version 15.0.07. Available in the usual places.
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Sat Mar 07, 2020 7:13 am

Hi Marcus,

Sorry for the late reply, been busy for the past few days. This is great!

I have a couple of additional questions:

• Is it possible to specify a chrome user profile on ChromeStart? (So I can launch a profile with pre-configured addons etc)

• Is it possible to specify a specific Chrome based browser for ChromeDriver to control? eg:
options.setBinary("C:\\Program Files (x86)\BraveSoftware\Brave-Browser\Application\brave.exe");

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Sun Mar 08, 2020 12:49 pm

Also is there a submit function we can use on form elements? eg:

Code: Select all

ChromeElementAction>session_id,formfield_1,submit
or a sendkeys function:

Code: Select all

ChromeElementAction>session_id,formfield_1,send_keys(Keys.ENTER)
A method of scrolling a specific element would also be very useful:

Code: Select all

ChromeFindElements>session_id,css selector,TabbedContent,Tabs
ChromeElementAction>session_id,Tabs_1,scrollDown,100
Last edited by NickD on Sun Mar 08, 2020 1:37 pm, edited 1 time in total.

User avatar
Marcus Tettmar
Site Admin
Posts: 7251
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by Marcus Tettmar » Sun Mar 08, 2020 1:31 pm

Yes ChromeElementAction can take “submit” or “click” actions.

If you specify submit on any form element it should submit the form. See sample script which does that - submits the text box - instead of click the button.

Send keys is exactly what ChromeSetElementValue does.

See docs:

https://www.mjtnet.com/manuals/v15/HTML ... tions.html

Specifying profile/chrome exe not possible at present. Will add those to wish list.
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Sun Mar 08, 2020 1:47 pm

Thanks Marcus,

I edited my last post and added a suggestion for supporting scroll on targeted elements, see above.
(This will be useful for loading data in infinite scroll divs etc)
Will continue to add suggestions if I come across any more features that I think will be beneficial.

User avatar
Marcus Tettmar
Site Admin
Posts: 7251
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by Marcus Tettmar » Sun Mar 08, 2020 5:07 pm

Thanks for that.

Regarding sending keystrokes, it may not have been obvious that you can also send non-character keys like Return, Backspace etc. So have updated the docs:

Key Codes for Edge and Chrome
ChromeSetElementValue
EdgeSetElementValue

So e.g.:

Code: Select all

Let>CHROMEDRIVER_EXE=c:\chromedriver.exe

//start a Chrome session
ChromeStart>session_id

//navigate to the form
Let>strURL=https://www.mjtnet.com/contact-form/formpage.html
ChromeNavigate>session_id,url,strURL

//find the message textarea
ChromeFindElements>session_id,name,message,message_elements

//send some text
ChromeSetElementValue>session_id,message_elements_1,First line

//press enter twice
ChromeSetElementValue>session_id,message_elements_1,\uE006
ChromeSetElementValue>session_id,message_elements_1,\uE006

//more text
ChromeSetElementValue>session_id,message_elements_1,Second liKE

//backspace twice
ChromeSetElementValue>session_id,message_elements_1,\uE003
ChromeSetElementValue>session_id,message_elements_1,\uE003

//fixed text
ChromeSetElementValue>session_id,message_elements_1,ne
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Sun Mar 08, 2020 6:39 pm

Thanks Marcus, I did not realise that.

Code: Select all

ChromeElementAction>session_id,chatbox_1,submit
Was not working for me, but:

Code: Select all

ChromeSetElementValue>session_id,chatbox_1,\uE006
Works fine :D

Does SetElementValue support the same keycodes as Selenium then?

User avatar
Marcus Tettmar
Site Admin
Posts: 7251
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by Marcus Tettmar » Sun Mar 08, 2020 6:43 pm

Macro Scheduler is using EdgeDriver/ChromeDriver which are both WebDriver implementations. And the key codes are defined by the WebDriver specs.
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Sun Mar 08, 2020 9:12 pm

Ran into a problem using the code injection SRT you posted due to targeting limitations. The Page I am trying to automate looks like this:

Code: Select all

<html>
<head>
</head>
<body>
<div id="wrap">
<div class="blah blah">...</div>
<div class="blah blah">...</div>
<div class="blah blah">...</div>
<div class="my_target">...</div>
<div class="blah blah">...</div>
</div>
</body>
</html>
I can inject a button like this:

Code: Select all

GoSub>ExecuteJS,session_id,document.getElementById(\"%TargetDiv%\").innerHTML=\"%MyCode%\"
But then I loose all the nested divs which contain content I need to interact with at later stages of the script.

I tried targeting with document.getElementsByClassName and document.getElementsByTagName, but neither worked.

Because the content is dynamic, the number of closing div tags can change so it's not practical to use Regex to pull the div html I need then use StringReplace to insert my code (as I am not sure where the div will end).

Is there a way to replace all the source code at once?, so I could use:

Code: Select all

ChromeGetInfo>session_id,source,theSource
StringReplace>theSource,<body>,<body>%MyCode%,theSource
I would also like to suggest the addition of innerHTML to ChromeGetElementData>:

Code: Select all

ChromeFindElements>session_id,id,TargetDiv,InjectionDiv
ChromeGetElementData>session_id,InjectionDiv_1,innerHTML,InjectionDivHTML
Not only will this be useful when replacing HTML, it will also be very useful for scraping applications where css classes are the only indication of required information.

User avatar
Marcus Tettmar
Site Admin
Posts: 7251
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by Marcus Tettmar » Mon Mar 09, 2020 2:46 pm

Your JS will change ALL the html to whatever you have in the MyCode variable.

Unless you have already retrieved the source, added the new html you want appended and concatenated both into MyCode then you will be overwriting everything.

So in your example if you are setting innerHTML of the wrap DIV to just a button then all the other divs and stuff inside the wrap DIV will disappear.

Surely you want to do this:

Code: Select all

document.getElementById(\"%TargetDiv%\").innerHTML=document.getElementById(\"%TargetDiv%\").innerHTML + \"%MyCode%\"
or shorter version using +=

Code: Select all

document.getElementById(\"%TargetDiv%\").innerHTML+=\"%MyCode%\"
E.g.: this adds a paragraph containing an input tag to the end of what is already there ...

Code: Select all

GoSub>ExecuteJS,session_id,document.getElementById(\"wrap\").innerHTML+=\"<p><input type=submit value=try></p>\"
To change entire body source you could do:

Code: Select all

GoSub>ExecuteJS,session_id,document.body.innerHTML=\"<p>entire page replaced with this!</p>\"
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Mon Mar 09, 2020 5:43 pm

Ahaa!

Thanks for the explanation Marcus :D
Did not realise I was overwriting everything, as the only divs I could target (with an id) were essentially wrapping the whole page anyway.

NickD
Pro Scripter
Posts: 53
Joined: Fri Sep 23, 2016 2:17 pm

Re: Chrome Driver - Traverse Iframes & Insert HTML

Post by NickD » Tue Mar 10, 2020 2:47 pm

Still struggling with this...

Code: Select all

GoSub>ExecuteJS,session_id,document.body.innerHTML=\"<p>blah blah</p>\"
Works almost as expected, although instead of replacing the entire page, it actually just replaces everything inside the page body tag.

Here is my code, which uses regex to extract the body tag and insert my desired content (an anchor id and button to scroll to this new anchor, which will hopefully allow me to load all content in an infinite scroll div before scraping the data, analysing, and interacting with it) the modal window displays the modified html as expected, but it does not get injected into the page, any ideas? (Will PM you the URL Marcus).

Code: Select all

  // Get Source HTML
  ChromeGetInfo>session_id,source,pageSource

  // Extract body tag & content
  Let>pattern=<body>.*\s*<\/body>
  RegEx>pattern,pageSource,0,body_array,body_matches,0

  // Verify & Exit if False
  If>body_matches>0
  Let>pageSource=body_array_1
  Else
  If>body_matches=0
  MDL>404 Body not Found!
  Exit>0
  Endif
  Endif


  // Define Custom Button
  Let>MyCode=<div style="width:100%; height:60px; margin:0px auto; position:absolute; top:0px; left:0px;"><button id="custom-btn" onclick="window.location.hash='infinite-loading-container' "; style="padding:14px 16px; background-color:transparent; border:1px solid #ffd300; color:#ffd300;">Injected Button</button></div>
  
  // Regex Replace (Add Button Code to Source)
  Let>pattern=<body>
  RegEx>pattern,pageSource,1,matches,num,1,<body>%MyCode%,pageSource
  
  // Regex Replace (Add Anchor Link ID to Source)
  Let>pattern=<div data-v-358985eb="" class="infinite-loading-container">
  Let>anchorLinkID=<div data-v-358985eb="" class="infinite-loading-container" id="infinite-loading-container">
  RegEx>pattern,pageSource,1,matches,num,1,%anchorLinkID%,pageSource
  
  MDL>pageSource
  
  GoSub>ExecuteJS,session_id,document.body.innerHTML=\"%pageSource%\"

Post Reply
Sign up to our newsletter for free automation tips, tricks & discounts