Results via csv and webrecorder - now how to combine?

[ZeitenWanderer]

Hi,

for once I have a somewhat unspecific question: I do manage to send data from a csv file to a website (thanks for the xltowebform how-to-zip in the support section). And it works to send content from different columns or rows to different fields.

But now, I would like to pull the desired data out of the website (no idea how to that at the moment) and write it into the third column, right behind the two fields, which were sent to the website.

- How to retrieve the data? Would one use web recorder?

- How would one address the csv file and the proper cell? This has been discussed quite often, nevertheless I do not get it working for me. Any closer hints, which actually deal with a way at least close to it? There are many code snippets and hints, but I was unable to find one, which goes directly back into that csv file. Or is this a false approach at all?

[Dorian (MJT support)]

Hi ZeitenWanderer,

I think this KB article will help you get started with extracting the data you need.

What methods have you tried so far for writing it back to your CSV?

[Marcus Tettmar]

CSV is a flat file format. You can't "insert" data, you can only append by writing a new line to the end. So you can't just rewrite specific cells/columns without rewriting the entire file. If it's a new record, no problem, just add it to end of file. But if you want to UPDATE the file then unless you want to recreate the entire file CSV is not the way to go.

Instead, if you have Excel installed, you can use the built in XL functions. Excel can work with CSV files.

Alternatively you might want to consider whether or not CSV makes sense for this kind of thing. Unless you are forced to use CSV you would be better off using a database if you want to update specific values.

[ZeitenWanderer]

Thanks for the first hint, that is a neat help!

About the database: Actually the data comes out of an Access database, I had started with csv, since that was easy to follow ...
I am still a beginner with this.

So, that rewrites the last question: Would there be an example, a beginner could follow up?

[Marcus Tettmar]

Well you can update an Access database directly. No need to export to CSV.

This would be a good place to start:
http://help.mjtnet.com/article/20-using ... -functions

[ZeitenWanderer]

Well, I used the wonderful IE Element Wizard to extract text from an ID: results_avg_price - but it inserts code containing the website address including that specific item, where I had expected it to grab only the value presented in that ID.

Reading the comment, one can define where the text would appear, but I am at loss to understand how to modify the code, that I would grab in a loop the latest value in that ID.

[snippet=]If>numRows>1
Let>row=1
//loop through rows (start row 2, row 1 has headers)
Repeat>row
//get each item (column)
Let>row=row+1
XLGetCell>xlBook,Sheet1,row,1,strTitel
XLGetCell>xlBook,Sheet1,row,2,strAutor
//fill the form (calls FillForm subroutine)
GoSub>FillForm

//here we click on "Search"
IEClickTag>%IE[0]%,{""},{"search_header"},{"INPUT"},{"ID"},{"search_header_submit"},ie_res
IEFormSubmit>%IE[0]%,{""},{"search_header"},ie_res

IEWaitDocumentComplete>%IE[0]%,ie_res

// Change "O" to H to extract inner html, T for just text. O is outer html - i.e. everything. Matches returned in valueArray
IEGetTagsByAttrib>{"http://www.eurobuch.com/search_results. ... valueArray
[/snippet]

How would I have to modify IEGetTagsByAttrib> that I could use it in a loop?

[Marcus Tettmar]

Really hard for me to answer this without having the web page in front of me so that I can look at the HTML. If that isn't possible maybe you can use IE's developer tools to pull out the relevant section of HTML and paste it here (within CODE tags) so that I can take a look and advise.

Sometimes the data you want is amongst other data and not on it's own in it's own identifiable element. In that case you would need to further parse the data out using string manipulation/regex.