Skip to content

50.003 - UI Testing

Learning Outcomes

  1. Describe the objectives of GUI testing
  2. Describe different types of GUI testing
  3. Develop automated GUI unit testing
  4. Develop automated end-to-end testing

The user interface is a software component that collects user input data, translates them into system expected input and present the results to the users.

There are many forms of user interfaces, e.g. visual, auditory, haptic, multimodal, and etc. In this module we only focus on visual user interface.

In modern computer system, we find many kinds of visual user interfaces.

  1. Command line
  2. Graphical user interface
  3. Chat bot
  4. ...

In this module, we put our attention to graphic user interface only.

Objective of Graphical User Interface Testing

There are two main objectives of conducting graphical user interface tests.

  1. To ensure functional correctness.
  2. To ensure visual correctness.

Functional GUI Tests are developed and conducted to ensure that the GUI component is behaving according to the functional requirements. Visual GUI Tests is to ensure that the GUI component is displayed and rendered according to the visual effect (non-functional) requirements.

Mode of UI Testing

Manual testing

Since UI is built for collecting input from human users, one way to test the UI components manually. Manual UI testing is labour intensive, as it requires enumerous amount of human resource and it is often mundane and repetitive.

Automated testing

Thanks to modern frameworks, there are approaches that allow tester to capture the human tester's behavior as a script and automate the testing.

Module Based UI Testing

Recall the earlier unit, testing is to find faults. It is often easier to find and isolate the fault if the software is developed in a modular way.

Example - Component Testing React.js App using Jest

Let's recall our echo app example introduced in the earlier units.

The Echo Component consists of a NewMessageBar component and a MessageList component.

Let's recall the NewMessageBar component, for the ease of testing, we move the following code into a standalone file NewMessageBar.js in the src/ folder.

function NewMessageBar({message, onMessageChange, onSubmitClick}) {
    return (
        <div>
            <input type="text" aria-label="echo-message" placeholder="" 
                value={message} 
                onChange={(e) => {onMessageChange(e.target.value)}}>
            </input>
            <button onClick={onSubmitClick}> Submit </button>
        </div>
    )
}

We consider the following unit test case in Echo.test.js. 

test('Button is rendered in NewMessageBar', () => {
    render(<NewMessageBar />);
    const button = screen.getByText(/submit/i);
    expect(button).toBeInTheDocument();
});
The render() function takes a React Component and renders in the test environment. The screen object allows us to make reference to the screen rendered in the test environment. The getByText() method takes a regex pattern which search for the text "submit" in upper or lower cases. The result is stored in the variable button. Finally we assert that the button can be found in the rendered document. We argue that the above test case is leaning towards the visual test category.

We recall another component used in the Echo app, MessageList. 

function MessageList({messages}) {
    let rows = [];
    for (let i in messages) {
        rows.push(
            <tr key={messages[i].time}
                data-testid={messages[i].time}>
                <td>{messages[i].time}</td><td>{messages[i].msg}</td></tr>
        );
    }
    return (
        <table data-testid="message-list">
            <tbody>
                <tr><th>Date Time</th><th>Message</th></tr>
                {rows}
            </tbody>
        </table>
    )
}

For the ease of reference, we give some test ids to the data rows and the table. We now can derive the following case. 

test('No message is rendered in empty MessageList', () => {
    const msgs = [];
    render(<MessageList messages={msgs} />);
    const table = screen.getByTestId("message-list");
    expect(table).toBeInTheDocument(); // the table must be rendered.
    expect(table.firstElementChild.children.length == 1); // only contains the header row.
});

In the above test case, we render a MessageList component with an empty message list. We expect the table's tbody element should contains only one row, which is the the header row.

In the next test case, we instantiate a MessageList component with a singleton message list and expect that the table should contains the message. 

test('A message is rendered in a singleton MessageList', () => {
    const msgTxt = "hello";
    const msgTime = (new Date()).toString();
    const msg = { time : msgTime, msg: msgTxt };
    const msgs = [msg];
    render(<MessageList 
            messages={msgs} />);
    const table = screen.getByTestId("message-list");
    const row = screen.getByTestId(msgTime);
    expect(table).toBeInTheDocument(); // the table must be rendered.
    expect(row).toBeInTheDocument(); // the row must be rendered.
    expect(table.contains(row));
});

Note that both of the test cases are functional tests.

Finally let's consider testing the Echo component. Recall that,

function Echo({http_addr}) {
    const [msgTxt, setMsgTxt] = useState("");
    function handleSubmitClick() {
        submitNewMessage();    
    }
    const [messages, setMessages] = useState([]);

    async function submitNewMessage() {
        const response = await fetch(`${http_addr}/echo/submit`,
        {
            method: 'POST',
            body: `msg=${msgTxt}`,
            headers: {
                'Content-type': 'application/x-www-form-urlencoded'
            }              
        });
        const text = await response.text();
        const json = JSON.parse(text);
        setMessages(json);
    }


    async function initMessages() {
        const response = await fetch(`${http_addr}/echo/all`);
        const text = await response.text();
        const json = JSON.parse(text);
        setMessages(json);
    }

    useEffect( () => {
        initMessages();
    }, []);

    return (
        <div>
            <NewMessageBar message={msgTxt} onMessageChange={setMsgTxt} onSubmitClick={handleSubmitClick}/>
            <MessageList messages={messages}/>
        </div>
    );
}

In the above code snippet, we abstract the backend API URL via the http_url variable, for ease of mocking the API host.

We define two test cases of the Echo component in the Echo.test.js test file as follows.

import { fireEvent, render, screen } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import './jest.polyfills'
import {http, HttpResponse} from 'msw'; 
import {setupServer} from 'msw/node';
import Echo from './Echo';

const server = setupServer(
    http.get('/echo/all', () => {
      return HttpResponse.json(
        [{'msg':'hello','time':'2024-06-24T04:24:49.000Z'}])
    }),
    http.post('/echo/submit', () => {
        return HttpResponse.json(
          [{msg:'hello',time:'2024-06-24T04:24:49.000Z'},
           {msg:'bye', time:'2024-06-25T00:12:30.000Z'}
          ])
      }),
    )

beforeAll(() => server.listen())
afterEach(() => server.resetHandlers())
afterAll(() => server.close())


describe("testing Echo component", () => {
    beforeAll(() => server.listen())
    afterEach(() => server.resetHandlers())
    afterAll(() => server.close())
    // some tests here 
    test('testing initMessage() in Echo', async () => {
        render(<Echo http_addr='' />);
        const msgTime = '2024-06-24T04:24:49.000Z';
        const table = await screen.findByTestId("message-list");
        const row = await screen.findByTestId(msgTime);
        expect(table).toBeInTheDocument(); // the table must be rendered.
        expect(row).toBeInTheDocument(); // the row must be rendered.
        expect(table.contains(row));
    });

    test('testing submitNewMessage() in Echo', async () => {
        const msgTxt = "bye";
        render(<Echo http_addr='' />);
        const textbox = screen.getByLabelText('echo-message');
        const submitButton = screen.getByText(/submit/i);
        fireEvent.change(textbox, {target: {value: msgTxt}});
        userEvent.click(submitButton);
        expect(textbox.value).toBe(msgTxt);
        const msgTime = '2024-06-25T00:12:30.000Z';
        const table = await screen.findByTestId("message-list");
        const row = await screen.findByTestId(msgTime);
        expect(table).toBeInTheDocument(); // the table must be rendered.
        expect(row).toBeInTheDocument(); // the row must be rendered.
        expect(table.contains(row));
    })
})

As we want to isolate the test from the backend API, we need to mock the API calls. This is achieved by using the library msw.

Due to some API deprecation, to import msw with versions of Jest and React we are currently using we need to introduce some extra configuration for Jest in jest.polyfills. For details setup, please refer to this file.

setupServer() functions allows us to define a mock backend API server with the required endpoints. The first test case 'testing initMessage() in Echo' is to test the first rendering of the Echo() component. We want to ensure that the table contains a row with the message returned by the mocked API /echo/all.

Note that findByTestId() is the async version of getByTestId(), it waits for the previous promise to be fully resolved before assigning the result to the LHS variable.

The second test case 'testing submitNewMessage() in Echo' is to test the user action of entering the message "bye" in the textbox and clicking the submit button. We want to assert that the table contains a row with the new message returned by the mocked API /echo/submit.

Reference

  • https://testing-library.com/docs/
  • https://testing-library.com/docs/react-testing-library/example-intro

Cohort Exercise (Non graded)

Continue with your solution to Cohort Exercise 8 question 2, develop unit test cases for the React components found in the HR app.

UI-based end-to-end testing

An end-to-end (E2E) testing is to assess the built system's feature from the user's perspective. All the components in the system supporting that particular feature are actively tested, i.e. no mocking.
An end-to-end test case is often derived directly from a use case.

To automate an end-to-end test, we can use test framework like Jest. For simple system feature like our Echo App. We could define our test as follows (similar to the Echo.test.js except that we don't require the mocking.)

function getRandomInt(max) {
  return Math.floor(Math.random() * max);
}

test('End-to-end testing on App', async () => {
  const msgTxt = "test message" + getRandomInt(1000);
  render(<App />);
  const textbox = screen.getByLabelText('echo-message');
  const submitButton = screen.getByText(/submit/i); 
  fireEvent.change(textbox, {target: {value: msgTxt}});
  await userEvent.click(submitButton);
  expect(textbox.value).toBe(msgTxt);
  await waitFor( () => {
      const table = screen.getByTestId("message-list");
      fireEvent.change(textbox, {target: {value: ''}});
      const text = screen.getByText(msgTxt);
      expect(table).toBeInTheDocument(); // the table must be rendered.
      expect(text).toBeInTheDocument(); // the text must be rendered.
      expect(table.contains(text));
  }); 
})

In the above test case, we use Jest to simulate the user's action sequence

  1. randomly generate a text message.
  2. click on the submit button.
  3. check that the message is stored by the system database and returned in the message list.

For real world application, a use case is often much more sophiscated than the above example, and might requiring more than a single UI. It is more effective to perform E2E testing using web browser simulation frameworks such as Selenium, Puppeteer, Cypress and etc.

You are strongly encouraged to explore and incoporate one of the following in your project.

References

  • Selenium https://www.youtube.com/watch?v=BQ-9e13kJ58
  • Cypress https://www.youtube.com/watch?v=jX3v3N6oN5M
  • Puppeteer https://www.youtube.com/watch?v=GB4oJ1Ru1Nk