March 28, 2018

Will robotic process automation take your job away?

Filed under: Automation,General,Macro Recorder — Marcus Tettmar @ 2:25 pm

TLDR: No! That’s not what we’re here for. Robotic process automation is here to help you.

We often get called in to help a company automate a process. Many times this involves automating something that an end-user does. To make macros that make their job easier, less painful, and make them more productive.

Initial Resistance to Change

Sometimes, at the outset, we (or the IT department proposing Macro Scheduler as a solution) detect a little resistance from the end user. If not properly briefed they may be concerned that what we want to do is take work away from them, and they may be concerned that the macros, or software robots, are about to take their jobs away.

Epiphany!

Nothing could be further from the truth and by the time we’re done, the end users love us to bits and usually give us a long list of other things they’d like us to automate! Often they discover how easy it is to do it themselves with tools like the Macro Recorder and other code wizards.

I’ve written thousands of macros for hundreds of companies. I’ve never worked on anything that was designed to take a job away from an employee, other than one the employee didn’t like doing and didn’t have time to do without it impacting on their core purpose. In every case the macros we wrote were there to make the employees’ jobs easier, or to allow them to be more productive in more important areas. For example: removing a tedious admin task from an already overstretched clinician, allowing them to see more patients.

I suppose you could argue that the many over-night data-entry style macros we’ve written could have been done instead by a team of data entry personnel. But such a team didn’t exist to start with, or would have been cost prohibitive, or too error-prone to be considered in the first place.

Improving Productivity

So to answer the question, no, I don’t believe robotic process automation is about taking jobs away. It’s not about replacing staff. In my 21 years of automating user interfaces this isn’t what I’ve seen. Instead, I’ve been rewarded with making people’s lives easier, removing cumbersome, painful, processes, or using macros to simplify and improve an existing process.

March 23, 2018

Screen Scraping with Macro Scheduler [Update]

Filed under: Automation — Marcus Tettmar @ 9:55 pm

What is Screen Scraping?

Screen Scraping is a term used to describe the process of a computer program or macro extracting data from the display output of another application. Rather than parsing data from the database or data files belonging to an application, Screen Scraping pulls the data from the screen itself, extracting data that was intended to be displayed to the end-user as opposed to data designed for output to another application or database.  

Screen Scraping is necessary when there is a need to access the information displayed by the application but there is no method provided to access it behind the scenes. The database or data files may not be accessible, or may be undocumented or proprietary and therefore cannot be parsed easily; the costs associated with interacting with the database may be too high; or the license agreement or warranty prohibits it. In the case of legacy systems that are no longer supported there may be no knowledge of the data structures, or the technology used is no longer compatible with current technology. In these cases we are resorted to extracting the data from the screen – from the windows of the application.

The term Screen Scraping probably originates from the era of computer terminals when you could connect the terminal output of a computer to an input port on another and therefore record the screen data.  

Screen Scraping Methods

There are a number of ways we can retrieve information from the screen using Macro Scheduler, depending on the type of application the data is in.

Screen Scraping Web Applications

Applications like Macro Scheduler’s WebRecorder can access the data and objects inside an Interner Explorer window and can therefore be used to extract the data.  Technically speaking I would not call this screen scraping since WebRecorder is using an API interface provided by Internet Explorer, but the process of extracting information from web sites is commonly refered to as Screen Scraping.  With WebRecorder we can use the ExtractTag wizard to create code that extracts the text from a particular element in the page.   While WebRecorder is the easiest way to do it, it is also possible to automate IE and extract data from web pages by using VBScript. The following forum posts may help:

Automate Internet Explorer with OLE/ActiveX
Automate web forms with IE
HTTP GET and POST using VBscript

Screen Scraping Microsoft Office Applications

Microsoft Office Applications, like Internet Explorer, have a COM interface that allows scripts to manipulate them and access the data held within them.  Again, not really scraping data from the screen itself, as you are getting it directly from a programming interface. There are a number of examples in the forums and blog archives and also some sample scripts that come with Macro Scheduler which demonstrate how to automate Office applications and retrieve data from them.  

Working with Excel

Screen Scraping Regular Windows Applications

Most other applications don’t offer a scripting interface like MS Office or Internet Explorer.  This is where we really need to work directly with the screen.   There are a number of ways we can do this kind of Screen Scraping with Macro Scheduler.

Screen Scraping via Optical Character Recognition

Macro Scheduler 14.4 includes some really neat functions which make it really easy to OCR a portion of the screen:

  • OCRScreen
  • OCRWindow
  • OCRArea

The first of these is the simplest. It simply scans the entire screen and returns all the text it can recognise. Of course this is also the slowest as it has to perform OCR against the entire screen. OCRWindow takes a window title and scans only the area of the screen where that window appears. This is nice and simple and a good compromise if the window isn’t too large. Finally, OCRArea can be given a rectangular screen region (X1,Y1,X2,Y2). You could use FindObject to find the coordinates of a specific UI object and pass those coordinates to OCRArea if you want to narrow things right down.

The Text Capture Functions

Macro Scheduler includes some Text Capture functions which can be used to extract text from a given window, rectangular screen area or screen point.  These functions use low level system hooks which monitor applications calling the various “TextOut” functions that Windows uses to output text to the screen.  By doing so they are able to capture this text.  The Text Capture functions return the text to  a variable which you can then use as needed.  

However, a few applications don’t use the Windows built-in functions to create and output text.  Don’t worry –Most do, but a few use their own techniques.  When you realise that text on the screen is just a sequence of small dots, if the application programmer decided to build his own routine to assemble text from dots rather than calling the Windows functions which already do that for you, you’re not going to be able to capture it.

The text capture functions and their limitations are explained here.  There is an example application, here, created with Macro Scheduler, which you can use to determine whether or not the text you want to capture can be captured using the text capture functions.

http://www.mjtnet.com/blog/2007/12/12/capturing-screen-text/
http://www.mjtnet.com/blog/2008/01/03/screen-scrape-text-capture-example/

Using the Clipboard for Screen Scraping

If the text you want to capture is selectable then you can use the clipboard to retrieve it.  A Macro Scheduler macro can send the keystrokes necessary to highlight and copy the text to the clipboard and then use the GetClipboard function to retrieve that text to a variable.  This is far less elegant than using the Text Capture functions but might be necessary if the application concerned is not utlising any of the Windows text out functions to create the text.

SetFocus>Notepad*
//Select ALL
Press CTRL
Send>a
Release CTRL
//Copy to clipboard
Press CTRL
Send>c
Release CTRL

//Get and display the data
WaitClipboard
GetClipboard>theData
MessageModal>theData

March 15, 2018

Saving You Time and Money With Robotic Process Automation

Filed under: Automation,General,Macro Recorder — Marcus Tettmar @ 3:24 pm

Before Macro Scheduler existed I was a junior member of an IT department. As part of my job I built small tools to automate specific tasks. At the time I was using VB. Each time I had to automate a task I had to reinvent the wheel a little.

The whole point of Macro Scheduler was to simplify the task of building automation routines, or software robots. To avoid having to reinvent the wheel.

There’s no reason why you can’t automate your Excel-to-SAP/WEB/ERP/ACME-Desktop-App by writing code using C# or C++. But it’s going to take you longer than using Macro Scheduler. Macro Scheduler functions like “SendText” and “UISetValue” encapsulate some pretty low level and quite convoluted code. The code wizards and macro recorders which help you use them are even more complicated.

One of the main purposes of Macro Scheduler is therefore to enable people to automate things more quickly and more easily than could be done with traditional programming tools. It makes it possible for non-programmers but also simplifies and speeds up automation for developers.

Over the last 21 years – yes 21, Macro Scheduler was first launched in 1997! – we’ve helped people with a lot of automation tasks. We offer consultancy and have been into people’s offices and also helped over the phone and remote desktop. Most routines take us a few hours to create a macro for. Some take a day or two. Rarely do we need to spend more than three days on one process, though there are some projects which involve a series of automation routines that may therefore take longer or be done over a few sessions.

To do the job from scratch with C#, C++ or VB might take weeks. Many people who approach us seem to be imagining that to automate their task may take days or weeks. They are often very surprised when we tell them it’s a few hours not a few days.

We are all about saving time and money. That’s what our tools do and it’s why we built them. Our tools mean you don’t have to pay developers lots of money to spend weeks or months building custom solutions.

The only down side to this from our point of view is that we routinely disappoint large consultancy businesses and potential partners who are used to selling IT contracts worth hundreds of thousands. They approach us thinking that there’s a huge opportunity and that we’ll pay them a large cut of a big consultancy project.

But rarely does a job require so much time, and that’s the reason we’re here. Sometimes I sense these people want us to “flesh” projects out. They think we’re “too good” at what we do. We should slow down, make things more complicated and thus charge for more time.

But that isn’t us. That’s not why we’re here or why we created Macro Scheduler. The whole ethos of Macro Scheduler and MJT Net Ltd is to save time, to find more efficient, less expensive ways of doing things that were once thought impossible or too expensive to do, and to enable people to automate without specialised knowledge.

That said we’re happy to work with companies on an on-going basis. I have found that most businesses approach us with one specific routine in mind and then when they see what can be done they realise how it can be applied across the organisation in other departments, for other teams. Saving a team one hour a week may not seem like much, but do that for 500 other teams and it adds up to a huge efficiency saving for the entire business.

If you would like to talk to us to find out how we could help your business, whether on an ongoing basis or just for a one-off job, please drop me a line here.

March 14, 2018

Macro Scheduler 14.4.07 – Improved HTTPS Support

Filed under: Announcements — Marcus Tettmar @ 7:04 pm

We have today made Macro Scheduler build 14.4.07 available.

This release effects only the HTTPRequest function when using SSL. It addresses an issue where some sites that used SNI (Server Name Identification) would fail to load.

We’ve also made TLS1.2 the default security protocol so that you don’t have to set it explicitly. You can still set a lower version (TLS1 or TLS1.1) if you need to override it by setting TLS_VER to 1 or 11.

Very few sites now use SSLv3 due to its vulnerabilities. Most now require TLS1.2. However, if you MUST use one of the older SSL versions instead of TLS you can do so by setting HTTPS_SSLVER to one of SSLV2, SSLV23, SSL3 or ALL.

It is also no longer necessary to set the HTTP_SSL variable as HTTPRequest will enable it if the URL starts with HTTPS.

So all that is needed for retrieving a page via SSL is:

HTTPRequest>https://www.mjtnet.com/,,GET,,strResponse

Finally, the default for HTTP_USERAGENT is now “Macro Scheduler (www.mjtnet.com)”. We’ve done this because we discovered that the default value set internally by the development library we use is often filtered out by administrators, causing requests to fail (lots of other apps use the same library). You can of course set it to whatever you want.

March 1, 2018

Macro Scheduler 14.4.05 Update Available

Filed under: Announcements — Marcus Tettmar @ 9:46 am

Macro Scheduler 14.4.05 maintenance release is now available for download from the usual places.

For details of changes please see the history list here.

Registered Updates | Trial Downloads | New Licenses

January 2, 2018

Happy New Year – Welcome to 2018!

Filed under: Announcements,General — Marcus Tettmar @ 10:16 am

Just a quick note to say Happy New Year to all our customers. This year will be our 21st year of selling Macro Scheduler. I think that’s pretty amazing. Thanks for all your support over the years and for continuing to use Macro Scheduler. It’s always great to hear how Macro Scheduler helps in your work and business processes, so please continue to keep in touch and let us know how it’s helping you. Here’s to a happy and healthy 2018!

October 29, 2017

Sending Email via Office 365

Filed under: Scripting — Marcus Tettmar @ 6:23 pm

We’ve had a few questions lately about how to send email using the SMTPSendMail function via the Office 365 SMTP server.

Until Macro Scheduler 14.4.02 which we have released today this wasn’t possible as explicit TLS was required and SMTPSendMail did not support it. We have now added this. So as long as you have updated to Macro Scheduler 14.4.02 (and you have a valid Office 365 account) here’s the code you need to send an email via the Office 365 mail server:

Let>server=smtp.office365.com
Let>SMTP_AUTH=1
Let>SMTP_USERID=YOUR_OFFICE365_USERID
Let>SMTP_PASSWORD=YOUR_OFFICE365_PASSWORD
Let>SMTP_PORT=587
Let>SMTP_SSL=1
Let>SMTP_USETLS=3
SMTPSendMail>RECIPIENT_EMAIL,server,YOUR_OFFICE365_EMAIL,YOUR_NAME,SUBJECT HERE,BODY TEXT,

See also:
http://help.mjtnet.com/article/4-sending-retrieving-emails-via-gmail-ssl

Macro Scheduler 14.4.02 Available

Filed under: Announcements — Marcus Tettmar @ 6:17 pm

Macro Scheduler 14.4.02 maintenance release is now available for download from the usual places.

For details of changes please see the history list here.

Registered Updates | Trial Downloads | New Licenses

October 11, 2017

What is Robotic Process Automation (RPA)?

Filed under: General — Marcus Tettmar @ 2:32 pm

Our longstanding customers will know we’ve never really been into jargon here at MJT. Jargon in the IT industry tends to come and go. And there’s something quite amusing about a term that pops up and is touted as being some amazing new thing when it simply describes something we’ve been doing here at MJT Net Ltd for 20 years! On the other hand any term that helps to describe what we do is useful and I suppose it’s nice to know that finally, after all this time, what we do in our little niche of the IT world now has a name: Robotic Process Automation or RPA.

So those of you who have been using Macro Scheduler for a while know what RPA is, even if you’ve never called it that.

Robotic Process Automation tools allow you to create “bots” (we’ve always called them macros) to automate business processes (often by simulating a user). With RPA we automate repetitive tasks – taking the robot out of the human – freeing people up for more value-adding activities.

Our customers have been automating their processes robotically for years, ever since Macro Scheduler was first released way back in 1997.

October 6, 2017

Case Study: Royal National Institute of Blind People (RNIB)

Filed under: Success Stories — Marcus Tettmar @ 8:02 am

Customer Profile

The Royal National Institute of Blind People (RNIB) in London is a leading source of information on sight loss and the issues affecting blind and partially sighted people. Practical and emotional support from RNIB helps those who are losing their sight, blind, or partially sighted face the future with confidence.

Business Challenge

One of the programs offered by RNIB is the Talking Book Service. Talking Books are mastered in a format called DAISY, which allows someone with visual impairment to navigate through an audio book in much the same way as a sighted person would browse through a print book, such as by parts, chapters, page, and phrase.
Adding this amount of navigation, however, can require a lot of mouse clicks and keyboard commands. For example, adding the navigation controls for just one page requires up to ten actions to be performed. When you multiply that by the number of pages in a book, it quickly adds up to thousands of actions. Not only is this time-consuming, but it is also prone to user error, not to mention a potential cause of repetitive strain injury (RSI).

Oliver Johnson, Senior Audio Engineer at RNIB Talking Book Studios, knew there must be a more efficient system and started to research the possibility of automating the process. Mr. Johnson was looking for something that was powerful enough to solve the problem but not so complex that only an expert could use it.

Solution

Upon downloading MJT Net’s Macro Scheduler, Mr. Johnson was immediately struck by how easy it was to use. What surprised him was discovering all the other functions built into Macro Scheduler that he hadn’t thought of previously. Having access to image recognition, looping, database functions, and DDE commands suddenly opened up new possibilities and gave him ideas for other projects.

Mr. Johnson’s first task using Macro Scheduler was the script he wrote to add Page Navigation in DAISY to a Talking Book. In his words, “the result was quite astounding. Whereas before a member of staff would have to manually add the pages one at a time over the course of a couple of hours, now they could just say how many pages they wanted to add, click OK, and walk away. [Macro Scheduler] would do everything else.” This has saved RNIB hundreds of staff hours every month on just one task, thereby reducing costs and increasing productivity at the same time.

Another area where Macro Scheduler has proved invaluable is in facilitating the use of screen readers and screen magnifiers, which make applications and websites accessible to people with sight loss. Screen readers are not always able to detect navigational buttons on websites. While most screen readers allow you to create scripts to help make such programs a bit more accessible, they are unable to simplify or automate the process.

With the wide range of commands available within Macro Scheduler, Mr. Johnson has been able to “control programs that screen readers can’t easily work with; speed up the process so that staff with a visual impairment can work quickly, accurately and confidently; and automate processes so that staff can be freed up to work on other tasks while more mundane, repetitive, and time-consuming tasks are handled by macros.”

Results

RNIB Talking Book Studios has seen tangible benefits since beginning to use Macro Scheduler:

Increase in Productivity

Macro Scheduler has given Talking Book Studios the ability to automate several processes that previously used hundreds of staff hours. As a result, staff members have been freed up to work on other projects. In addition, Macro Scheduler has improved the usability of programs so that staff members with a visual impairment can work more efficiently and accurately.

The office produces tens of thousands of audio files every month, which are impossible to proof by hand. Macro Scheduler performs these checks every night to ensure that the files have the correct format, sample rate, and bit rate.

Reduction in Errors

Automation of navigation controls has eliminated user error when mastering Talking Books in DAISY. This keeps production on track and on time. Not only does Macro Scheduler enable Talking Books to perform quality checks on audio files, it also generates daily reports that flag any files that may require further investigation. This alone has reduced the number of errors by 50%.

Ease of Use

When asked what he likes best about Macro Scheduler, Mr. Johnson reports: “Macro Scheduler is so simple to use. The fact that someone like me, whose only experience of writing any form of code was rudimentary HTML, can use it straight out of the box, without reading through a manual first, speaks volumes.”

Customer Support

Mr. Johnson says that “the level of support from Marcus and the team has been some of the best I’ve ever come across. The forum is a wealth of information, and it is very active, so if you do require some assistance, help is never far away. On more than one occasion several members have gone out of their way to help me solve problems or suggest alternative and better ways of achieving things.”

About MJT Net

MJT Net Ltd develops and markets automation utilities and productivity software for business and home Windows users. Its flagship product, Macro Scheduler, was first released in 1997. With more than 30,000 companies and individuals utilizing Macro Scheduler technologies, MJT Net has become synonymous with software automation and automated testing.

« Newer PostsOlder Posts »