Web Automation

Technical support and scripting issues

Moderators: Dorian (MJT support), JRL

Post Reply
User avatar
PepsiHog
Automation Wizard
Posts: 511
Joined: Wed Apr 08, 2009 4:19 pm
Location: Florida

Web Automation

Post by PepsiHog » Wed Apr 11, 2018 7:23 pm

Hello Everyone,

I'd like to thank the MJTNet team for the help they gave me previously on this script. I need a little bit more help with something related to the same script.

The script below, as it is now, goes to IMDB.com and retrieves the first three names listed for the said movie and appends that to the name of the folder the movie is in. These movies are copies of DVDs that I own.

The list of names it retrieves is close to the top of the page. It lists the main actors in the movie. Further down the page is a larger list of actors that are in the movie. Below that list is a link to show all actors in the movie. It is this list (a list of all actors) that I am interested in retrieving.

My issue. I can not find, in the html code (when you press F12), the address that the link is connected to, to show all actors. I want to retrieve all the names of all the actors in the movie. I need to be able to change to the address of the link and then get the names. I can likely get the names from the list, but I need help getting to the page that displays them.

So far the script only opens the instance and then works with the HTML coding. I am trying to stay away from manually clicking the link. When I run the macro I have the IE window shrunk down to just a very small box in the corner. I do this manually, have not bothered to do it with scripting. Though probably should go ahead and do that. Anyway, I digress.

I just need to get the address of the link provided. I can likely manage with getting the list of actors once I get to that page. But if you are bored.....well, go ahead. I won't complain. (hee-hee)

Thanks A Whole Bunch!
PepsiHog

Code: Select all

let>Drive=E
let>JPath=%Drive%:\Done
OnEvent>Key_Down,VK113,0,DoExit

Dialog>Dialog1
object Dialog1: TForm
  Left = 679
  Top = 249
  Width = 492
  Height = 250
  HelpContext = 5000
  AutoScroll = True
  BorderIcons = [biSystemMenu]
  Caption = 'CustomDialog'
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'MS Sans Serif'
  Font.Style = []
  OldCreateOrder = True
  ShowHint = True
  OnTaskBar = False
  PixelsPerInch = 96
  TextHeight = 13
  object MSMemo1: tMSMemo
    Left = 0
    Top = 0
    Width = 476
    Height = 212
    ScrollBars = ssBoth
    Align = alClient
    TabOrder = 0
  end
end
EndDialog>Dialog1


AddDialogHandler>Dialog1,,OnClose,DoExit

//Show>Dialog1

let>GFL_TYPE=1
GetFileList>%JPath%\*.*,JFolders
Separate>JFolders,;,JFolder
ArraySort>JFolder

let>asd=0
let>MovieList={""}

While>asd<%JFolder_count%
let>asd=%asd%+1
let>FolderName=JFolder_%asd%
RegEx>(.*\(\d{4}\)).*,FolderName,0,match,nom,1,$1,Temp
StringReplace>Temp,%JPath%\,,Temp
let>MovieName=%Temp% - IMDb

//////////////////////////////////////////////////
// Google Search
let>Prefix=https://www.google.com/?gws_rd=ssl#q=
StringReplace>MovieName,',%27,MovieName
StringReplace>MovieName,space,+,MovieName
let>MovieName=%Prefix%%MovieName%
//////////////////////////////////////////////////

GoSub>DoGetStarsNames

ConCat>MovieList,Temp
ConCat>MovieList,Stars
ConCat>MovieList,crlf
SetDialogProperty>Dialog1,MSMemo1,Text,%MovieList%

let>MF_RENAME=1
let>NewName=%Temp%%Stars%
MoveFile>%FolderName%,%JPath%\%NewName%

EndWhile

WriteLn>J:\MovieStarsList.txt,wtf,%MovieList%

exit
*/
let>MovieName=Wyatt Earp (1994)



GoSub>DoGetStarsNames


srt>DoGetStarsNames

IECreate>IE[0]
IENavigate>IE[0],%MovieName%,res
IEWait>%MovieName%
wait>2

IEExtractTag>%IE[0]%,,BODY,0,1,the_html

let>BEGINNING=<h3
let>END=</div>
RegEx>(?<=%BEGINNING%).*?(?=%END%),the_html,0,TheURL,nm,0

let>BEGINNING=href="
let>END=">
RegEx>(?<=%BEGINNING%).*?(?=%END%),TheURL_1,0,NewURL,nm,0

CloseWindow>Internet Explorer*
wait>3

/////////////////////////////////////////////////////////////////

IECreate>IE[1]
IENavigate>IE[1],%NewURL_1%,res
IEWait>https://www.google.com*
wait>2

// Screen Capture for visual confirmation of correct URL.
//GetScreenRes>w,h
//ScreenCapture>0,0,w,h,J:\Pics\%MovieName%.bmp
//wait>1

IEExtractTag>%IE[1]%,,BODY,0,1,the_html

let>BEGINNING=Stars:</h4>
let>END=</div>
RegEx>(?<=%BEGINNING%).*?(?=%END%),the_html,0,TheURL,nm,0


let>BEGINNING=name">
let>END=</span>
RegEx>(?<=%BEGINNING%).*?(?=%END%),TheURL_1,0,StarsName,nm,0

let>Stars= %StarsName_1%, %StarsName_2%, %StarsName_3%

CloseWindow>Internet Explorer*
wait>3

END>DoGetStarsNames


srt>DoExit
  exit
END>DoExit



/*
<span itemprop="actors" itemtype="http://schema.org/Person" itemscope="">
<a href="/name/nm0000886?ref_=tt_ov_st_sm" itemprop="url"><span class="itemprop" itemprop="name">Warren Beatty</span></a>,             </span>
            <span itemprop="actors" itemtype="http://schema.org/Person" itemscope="">
<a href="/name/nm0000932?ref_=tt_ov_st_sm" itemprop="url"><span class="itemprop" itemprop="name">Halle Berry</span></a>,             </span>
            <span itemprop="actors" itemtype="http://schema.org/Person" itemscope="">
<a href="/name/nm0011129?ref_=tt_ov_st_sm" itemprop="url"><span class="itemprop" itemprop="name">Kimberly Deauna Adams</span></a>            </span>
            <span class="ghost">|</span>
<a href="fullcredits?ref_=tt_ov_st_sm">See full cast &amp; crew</a>&nbsp;»
    


srt>DoGoogleSearch
//https://www.google.com/?gws_rd=ssl#q=Ferris+Bueller%27s+Day+Off+(1986)-+IMDb
let>Prefix=https://www.google.com/?gws_rd=ssl#q=
StringReplace>MovieName,',%27,MovieName
StringReplace>MovieName,space,+,MovieName
let>MovieName=%Prefix%%MovieName%
//mdl>%MovieName%
//exit

END>DoGoogleSearch



 class="r"><a onmousedown="return rwt(this,'','','','1','AFQjCNFQME8bNf68vBCbjEuibh9_wsb35A','','0ahUKEwiymqCh6eDLAhVI8CYKHUomCHYQFggcMAA','','',event)" href="http://www.imdb.com/title/tt1156398/">Zombieland (2009) - IMDb</a></h3><div class="s"><div><div class="f kv _SWb" style="white-space: nowrap;"><cite class="_Rm">www.<b>imdb</b>.com/title/tt1156398/</cite><div class="action-menu ab_ctl"><a class="_Fmb ab_button" id="am-b0" role="button" aria-expanded="false" aria-haspopup="true" aria-label="Result details" href="#" data-ved="0ahUKEwiymqCh6eDLAhVI8CYKHUomCHYQ7B0IHTAA" jsaction="m.tdd;keydown:m.hbke;keypress:m.mskpe"><span class="mn-dwn-arw"></span></a><div tabindex="-1" class="action-menu-panel ab_dropdown" role="menu" data-ved="0ahUKEwiymqCh6eDLAhVI8CYKHUomCHYQqR8IHjAA" jsaction="keydown:m.hdke;mouseover:m.hdhne;mouseout:m.hdhue"><ol><li class="action-menu-item ab_dropdownitem" role="menuitem"><a class="fl" onmousedown="return rwt(this,'','','','1','AFQjCNGEpzoUbu7XJFBO-Da7bhwS7_m1Hg','','0ahUKEwiymqCh6eDLAhVI8CYKHUomCHYQIAgfMAA','','',event)" href="http://webcache.googleusercontent.com/search?q=cache:yTb3F26GnpMJ:www.imdb.com/title/tt1156398/+&amp;cd=1&amp;hl=en&amp;ct=clnk&amp;gl=us">Cached</a></li><li class="action-menu-item ab_dropdownitem" role="menuitem"><a class="fl" href="/search?biw=1324&amp;bih=685&amp;q=related:www.imdb.com/title/tt1156398/+Zombieland+(2009)+-+IMDb&amp;tbo=1&amp;sa=X&amp;ved=0ahUKEwiymqCh6eDLAhVI8CYKHUomCHYQHwggMAA">Similar</a></li></ol>
 href="http://www.imdb.com/title/tt1156398/">
 
 
 
        <span itemprop="actors" itemtype="http://schema.org/Person" itemscope="">
<a href="/name/nm0251986?ref_=tt_ov_st_sm" itemprop="url"><span class="itemprop" itemprop="name">Jesse Eisenberg</span></a>,             </span>
            <span itemprop="actors" itemtype="http://schema.org/Person" itemscope="">
<a href="/name/nm1297015?ref_=tt_ov_st_sm" itemprop="url"><span class="itemprop" itemprop="name">Emma Stone</span></a>,             </span>
            <span itemprop="actors" itemtype="http://schema.org/Person" itemscope="">
<a href="/name/nm0000437?ref_=tt_ov_st_sm" itemprop="url"><span class="itemprop" itemprop="name">Woody Harrelson</span></a>            </span>
            <span class="ghost">|</span>
<a href="fullcredits?ref_=tt_ov_st_sm">See full cast &amp; crew</a>&nbsp;»
    
Windows 7

PepsiHog. Yep! I drink LOTS of Pepsi (still..in 2021) AND enjoy programming. (That's my little piece of heaven!)

The immensity of the scope of possibilities within Macro Scheduler pushes the user beyond just macros!

hagchr
Automation Wizard
Posts: 327
Joined: Mon Jul 05, 2010 7:53 am
Location: Stockholm, Sweden

Re: Web Automation

Post by hagchr » Thu Apr 12, 2018 2:21 pm

Hi, It seems if you adjust the link of the movie then you can get to the full list directly, eg:

http://www.imdb.com/title/tt0076759/
Adjust to:
http://www.imdb.com/title/tt0076759/fullcredits

User avatar
PepsiHog
Automation Wizard
Posts: 511
Joined: Wed Apr 08, 2009 4:19 pm
Location: Florida

Re: Web Automation

Post by PepsiHog » Thu Apr 12, 2018 9:44 pm

Awesome! Thanks, hagchr! That's easy enough.
Windows 7

PepsiHog. Yep! I drink LOTS of Pepsi (still..in 2021) AND enjoy programming. (That's my little piece of heaven!)

The immensity of the scope of possibilities within Macro Scheduler pushes the user beyond just macros!

Post Reply
Sign up to our newsletter for free automation tips, tricks & discounts