背景

rerun

在基于robot framework框架進行自動化測試的時候，有時候測試用例可能由于一些外在原因?qū)е率。ㄈ缇W(wǎng)絡(luò)中斷，返回超時）等，并不是用例本身驗證有問題；而且這些場景有可能是偶發(fā)的，因此我們需要一種失敗用例重執(zhí)行的機制。
robot framework 官方并沒有提供類似retry等參數(shù)來配置失敗用例重執(zhí)行。僅僅提供了--rerunfailed參數(shù)對基于結(jié)果文件output.xml來選擇重新執(zhí)行失敗的用例。

實現(xiàn)

我們先寫一個簡單的，有50%概率運行失敗的測試腳本。

# 存放在tests目錄下
*** Settings ***
Library  String

*** Test Cases ***
stable_test
    should be true  ${True}
unstable_test
    ${bool} =  random_boolean
    should be true  ${bool}
    
*** Keywords ***
random_boolean
    ${nb_string} =  generate random string  1  [NUMBERS]
    ${nb_int} =  convert to integer  ${nb_string}
    Run keyword and return  evaluate  (${nb_int} % 2) == 0

重試機制如下：

# first execute all tests
pybot --output original.xml tests 
# then re-execute failing
pybot --rerunfailed original.xml --output rerun.xml tests 
# finally merge results
rebot --merge original.xml rerun.xml

我們查看日志輸出：

report

重試機制腳本化

我們編寫一個腳本launch_test_and_rerun.sh

#!/bin/bash

# clean previous output files
rm -f output/output.xml
rm -f output/rerun.xml
rm -f output/first_run_log.html
rm -f output/second_run_log.html

echo
echo "#######################################"
echo "# Running portfolio a first time      #"
echo "#######################################"
echo
pybot --outputdir output $@

# we stop the script here if all the tests were OK
if [ $? -eq 0 ]; then
        echo "we don't run the tests again as everything was OK on first try"
        exit 0
fi
# otherwise we go for another round with the failing tests

# we keep a copy of the first log file
cp output/log.html  output/first_run_log.html

# we launch the tests that failed
echo
echo "#######################################"
echo "# Running again the tests that failed #"
echo "#######################################"
echo
pybot --outputdir output --nostatusrc --rerunfailed output/output.xml --output rerun.xml $@
# Robot Framework generates file rerun.xml

# we keep a copy of the second log file
cp output/log.html  output/second_run_log.html

# Merging output files
echo
echo "########################"
echo "# Merging output files #"
echo "########################"
echo
rebot --nostatusrc --outputdir output --output output.xml --merge output/output.xml  output/rerun.xml
# Robot Framework generates a new output.xml

運行腳本查看執(zhí)行結(jié)果

 $ ./launch_test_and_rerun.sh test                                                                                                                     ?

#######################################
# Running portfolio a first time      #
#######################################

==============================================================================
Test
==============================================================================
Test.Demo
==============================================================================
stable_test                                                           | PASS |
------------------------------------------------------------------------------
unstable_test                                                         | FAIL |
'False' should be true.
------------------------------------------------------------------------------
Test.Demo                                                             | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Test                                                                  | FAIL |
2 critical tests, 1 passed, 1 failed
2 tests total, 1 passed, 1 failed
==============================================================================
Output:  /Users/wangyang/opg/demo/output/output.xml
Log:     /Users/wangyang/opg/demo/output/log.html
Report:  /Users/wangyang/opg/demo/output/report.html

#######################################
# Running again the tests that failed #
#######################################

==============================================================================
Test
==============================================================================
Test.Demo
==============================================================================
unstable_test                                                         | PASS |
------------------------------------------------------------------------------
Test.Demo                                                             | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Test                                                                  | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /Users/wangyang/opg/demo/output/rerun.xml
Log:     /Users/wangyang/opg/demo/output/log.html
Report:  /Users/wangyang/opg/demo/output/report.html

########################
# Merging output files #
########################

Output:  /Users/wangyang/opg/demo/output/output.xml
Log:     /Users/wangyang/opg/demo/output/log.html
Report:  /Users/wangyang/opg/demo/output/report.html

總結(jié)

通過修改robot framework框架代碼添加重試機制，會使框架變得復(fù)雜和不穩(wěn)定，因此官方也不推薦修改源碼方式增加失敗重試機制，所以我們需要在外部邏輯對失敗重試進行設(shè)計。