Dynamic Tables in Automation

This blog was originally posted on Medium.

I’m pretty new to automation. I’m a self taught lone tester who started out in automation basically trying anything. I previously worked as a developer, using Matlab, so code wasn’t new to me. Our team knew that we needed to get automation up and running to help with the regression efforts. I am only one person after all.

It was decided that the automation code would be written in Java as the developers would be able to read it and possibly help me if I ran into difficulty. I tinkered around a bit, understanding page objects and locators but not really getting to where I thought I should be. My company contracted in someone for a few days to help me get a proper automation suite up and running. Everything was looking rosy. Things were popping up on the screen, getting filled in, we had identified some page load problems and I had something to show my boss.

The contractor left and a short time later things were not looking so rosy. I was stuck. I had something that didn’t appear to be scalable. The contractor had assured me that this was the best process for our application but I soon discovered that it wasn’t. Maybe I didn’t explain our product well enough to them, maybe they always write their code like this or had never experienced this before. I keep retrospecting on this and I can’t come up with the answer. All I can say is that this was a fantastic learning curve for me and my team have been very patient as I tried to refactor all of the code.

A bit of a background on our application: it is a clinical trial simulation tool. You can read more about it here. To anyone reading this with a dev/tester background it basically means a LOT of dynamic tables. When I say a lot I mean a table on 90% of the pages for the 17 page objects I have declared so far. A thing of beauty for the user, a thing of horror for the newbie automator.

I felt like I couldn’t go back to the contractor. Let’s be honest, for a small company, the feeling is that they cost a lot and it can be hard to find a “good” one. I asked the developers to explain what the tables were and how they were created. They said our application created dynamic tables using AngularJS. Extensive googling of this didn’t net me any results (I probably used the wrong combination of search words). I asked for help in Ministry of Testing slack channel “Does anyone have any experience automating for dynamic tables?”. Andrew Morton and Richard Bradshaw replied. They had, success!

Andrew sent me code snippets which turned into functions which turned into lengthy discussions about how best to organise my automation. Andrews thinking was if you have something, like tables, that appear frequently in your application they should fall into a utility page object. That is to say that functions to search tables, locate cells of tables, etc. should have their own page object that would have a title “tableUtility” or similar. The table utility functions would then be called from the page objects rather than repeating code over and over in page objects to interact with tables.

Richard sent me a link to a blog post and some code on Github. This started a long slack thread with Richard and, at times, Andrew (mostly about lambda functions because it took me a long time to get the hang of them). I have to thank them both for their patience with me. I would have been lost without them. I think one of my favourite pieces of advice from Richard was: “Think about how you would search a table for information in every day life. You would know that you are looking for a customer so you would find the header for that and search the column for the customer you want. You can search a table in other ways so think about that before you try to understand the code.” At the time I was frustrated because I had this mental block of “I’ve done it all wrong and I’m costing my company money trying to fix it”. You even have the fear of “Will they sack me because this is a pretty big screw up”. I was learning (I still am-you never really stop) and I was too hard on myself, my company knew I was learning too and I could give valid justification for why the refactor needed to happen. With the patience of Andrew and Richard (and a lot of yoga) I managed to get something working. Now, not only do I have it working but I have expanded on the functionality from Richards code and I have it working well in Java.

I learned that my XPaths were actually the problem, not my lambda functions. Always start with the basics or you’ll spend ages searching the complex things and getting increasingly frustrated. You’ll then kick yourself if you discover it was a very simple thing that caused the problem.

So a simpler view of what I’ve been dealing with over the past few months. I’ve added some HTML below with an image of the table that it generates. This is a pretty simplified version of our tables!

Using w3schools with the above, I generated a table like this:

A 4 column table with headings: Company, Contact, Country, Actions. There are 6 rows of example entries for this table.

A much simpler version of the tables I’ve encountered!

So what was the problem? Why was our code not scalable? In some cases, the number of columns/rows on one page depended on the user selection on a previous page. Our previous locators looked like this:

I mean there isn’t anything wrong with this. It did work for one user journey through the system. The problem was that it wasn’t scalable because if there were more (or less) user entries on the previous page, then the XPaths changed. I learned this quite quickly. I was going to have to add a new locator and new method to use that locator for each combination I wanted to check. It did work but I thought there had to be an easier way.

This is where the code from Richard came in. To start with, I now just use the ID of the table in the page object and perform cell/row/column locations in the table utility. The below code calls a function in the table utility to find the table headers and return them as a list.

That is one of the easier examples but isn’t it tidy? It’s scalable too. If the number of headers on the table changes I don’t have to update the function or the XPaths. All I would have to update in that case would be the expected results for my automated check.

I have more examples of how to use each of the functions on my own Git repository if you want the java version of Richards code. I have added some code to mine to locate buttons in a cell. Our tables have different sets of actions, not all tables have the same actions buttons, so I’ve had to add some different functions for those. An example:

A 4 column table with headings: Label, Description, Status, Actions. There is 1 row of example entries for this table. The actions entry has 4 buttons nested within the cell.

Note the buttons in the far right cell.

XPaths for those buttons from left to right:

.//*[@id=’someTable']/tbody/tr/td[4]/div/a
.//*[@id=’someTable']/tbody/tr/td[4]/div/button[1]
.//*[@id=’someTable']/tbody/tr/td[4]/div/button[2]
.//*[@id=’someTable']/tbody/tr/td[4]/div/button[3]

To click the buttons you can use the code below in your page object. The headerToSearch would be “Actions”-you are looking in the Actions column for the buttons. The variables knownLabel and knownColumnHeader would be “Label” and “Use2Find” or “Description” and “Another column of known information you can use to find buttons”.

Some closing comments: watch your HTML tags, in particular tbody vs thead. Depending on how the HTML is written will dictate how you declare/handle the variables tableHeaders and tableBody. For example, in our application our header row, first column would have an XPath like this:

.//*[@id=’someTable’]/thead/tr/th[1]

The second row of the table, the one that appears under the header row, would have an XPath like this:

.//*[@id=’someTable’]/tbody/tr[1]/td[1]

I have encountered other tables whose XPath for the header was:

.//*[@id=’someTable’]/tbody/tr/th[1]

I missed this when I first examined the table and spent far too long trying to get to the root of the problem. I spoke to the developers about it and it’s something we’re all aware of going forward, so we will always have the format of the first XPath example.

If in doubt, always go back to basics.