Due to the increasing popularity of web applications, and the number of browsers and platforms on which such applications can be executed, cross-browser incompatibilities (XBIs) are becoming a serious concern for organizations that develop web-based software. Most of the techniques for XBI detection developed to date are either manual, and thus costly and error-prone, or partial and imprecise, and thus prone to generating both false positives and false negatives. To address these limitations of existing techniques, we developed X-PERT, a new automated, precise, and comprehensive approach for XBI detection. X-PERT combines several new and existing differencing techniques and is based on our findings from an extensive study of XBIs in real-world web applications. The key strength of our approach is that it handles each aspects of a web application using the differencing technique that is best suited to accurately detect XBIs related to that aspect. Our empirical evaluation shows that X-PERT is effective in detecting real-world XBIs, improves on the state of the art, and can provide useful support to developers for the diagnosis and (eventually) elimination of XBIs.

1. X"PERT:(Accurate(Identification((
of(Cross2Browser(Issues((
in(Web(Applications(
)
Shauvik Roy Choudhary, Mukul Prasad, Alessandro Orso
Labs of America
Partially supported by

2. Web(Applications9
2

3. 3

4. 4
Mozilla(Firefox) Internet(Explorer)

5. 5
Mozilla(Firefox) Internet(Explorer)

6. 6
Google(Chrome)

7. 7
Apple(Safari)

8. 8
Apple(Safari)Mozilla(Firefox)

9. 9
Mozilla(Firefox)

10. 10
Mozilla(Firefox)
Internet(Explorer)

11. 11
Mozilla(Firefox)
Internet(Explorer)

12. (9
Web(Developer)

13. (9
Web(Developer)

14. (9
Web(Developer)

15. (
Browser(versions(last(year9
9
9
Source:(StatCounter.com9

16. (
Browser(versions(last(year9
with(at(least(1%(market(share9
9
150mn(visits/month((StatCounter.com)9

20. Challenges(
9
Manual(inspection9
is(expensive9
DOM(differs(9
between(browsers9
Mimic(end(user’s9
perception(9
Produce(readable,(9
effective(&(actionable(9
reports9
Engineering9
Issues…9
Modern(apps(have9
many(dynamic(screens(9

21. Detection(of(
Cross2Browser(Issues((XBI)9
Web(Application9
12
My Awesome Site
Model(Generation9 Model(Comparison9
Error(Report9

22. Model(Definition((
and(Comparison9
WebDiff([ICSM’10])
Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])
Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](
Mesbah(and(Prasad9
Screen(Transition(Graph)

23. Model(Definition((
and(Comparison9
WebDiff([ICSM’10])
Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])
Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](
Mesbah(and(Prasad9
Screen(Model)
Screen(image(9
+(geometry9
DOM(Tree9

24. Model(Definition((
and(Comparison9
WebDiff([ICSM’10])
Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])
Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](
Mesbah(and(Prasad9

25. Study(of(Real(World((
Cross2Browser(Issues((XBIs)9
• Examined 100 websites
• Found 23 with XBIs
• XBI Types
o Behavioral XBIs
o Content XBIs
o Structural XBIs
((9%)
22%)
30%)
57%)
Text
Visual
Random(URL(Generator9

26. Limitations(of((
existing(techniques9
• Miss Structural XBIs
• Focus on symptoms rather than
causes
o Duplicate reports
o Low Precision

27. Example9

28. Example(
9

29. Example(
9

30. Example(
9
Behavior(XBI)
9%)

31. Example(
9

32. Example(
9
Content(XBI)
(Visual)(–(30%)

33. Example(
9
Structural(XBI)
57%)
Content(XBI)
(Visual)(–(30%)

34. Example(
9
Content(XBI)
(Visual)(–(30%)
Content(XBI)
(Text)(–(22%)
Structural(XBI)
57%)

35. Comprehensive(
XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text
Visual

36. Comprehensive(
XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text
Visual
Behavior(check(using(9
Graph(Isomorphism9
CrossCheck([ICST’12])
Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](
Mesbah(and(Prasad9

37. Comprehensive(
XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text
Visual
Text(Content)
check(using(9
String(equality9
WebDiff([ICSM’10])
Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])
Roy(Choudhary,(Prasad(and(Orso9
CrossT([ICSE’11](
Mesbah(and(Prasad9

38. Comprehensive(
XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs
Text
Visual
Visual(Content)
check(using(9
Image(Matching9
Computer9
Vision9
WebDiff([ICSM’10])
Roy(Choudhary(and(Orso9
CrossCheck([ICST’12])
Roy(Choudhary,(Prasad(and(Orso9

39. Detecting(Structural(XBIs(
9• Intuition:
Disruption in the relative
arrangement of web page
elements leads to Structural XBIs
• Idea:
Abstraction to represent relative
arrangement and compare it to
expose errors

40. Alignment(Graph((AG)(
9
• A graph capturing structural
relationships between elements
(rectangles) on the webpage

41. Alignment(Graph((AG)(
9
• A graph capturing structural
relationships between elements
(rectangles) on the webpage
• An AG has:
o Containment relationships
o Alignment relationships
o Attributes
above9
right2align9
left2align9

42. Step(1:(AG(Construction(
9

43. Step(1:(AG(Construction(
9

44. Step(1:(AG(Construction(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
19
29
39
49
59
69
79
89
99
109
119
129
139
149
159
169
179
189

45. Step(1:(AG(Construction(
a.(Containment(Relationship)
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
e(contains(e’(iff9
• ((x1(≤(x1’( y1(≤(y1’( x2(≥(x2’( y2(≥(y2’()(,(and9
• if((same(bounds(and((XPath(e)(≤(XPath(e’)(9
e9
e’9

46. Step(1:(AG(Construction(
a.(Containment(Relationship)
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2

47. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
b.(Alignment(Relationship)

48. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
b.(Alignment(Relationship)

49. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
b.(Alignment(Relationship)

50. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
c.(A[ributes)
Bugon(19 Bugon(29

51. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?(
)

52. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?(
)
• Bugon(1(is(on(the(left(of(Bugon(2)
x2(<(x1’9

53. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?(
)
• Bugon(1(is(on(the(left(of(Bugon(2)
• Bugon(1(and(Bugon(2(have(their(tops(aligned)
)
)
y1(=(y1’9

54. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
c.(A[ributes)
Bugon(19 Bugon(29
What(alignment(a[ributes(can(we(infer?(
)
• Bugon(1(is(on(the(left(of(Bugon(2)
• Bugon(1(and(Bugon(2(have(their(tops(aligned)
)
• Bugon(1(and(Bugon(2(have(their(bo[oms(aligned)
y2(=(y2’9

55. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
Step(1:(AG(Construction(
c.(A[ributes)
What(alignment(a[ributes(can(we(infer?(
)
• Bugon(1(is(on(the(left(of(Bugon(2(
9
• Bugon(1(and(Bugon(2(have(their(tops(aligned(
)
• Bugon(1(and(Bugon(2(have(their(bo[oms(aligned)
leftOf)
top"align)
bo[om"align)

56. Step(2:(Matching(AGs(
9

57. Step(2:(Matching(AGs(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align

58. Step(2:(Matching(AGs(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align

59. body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
Step(2:(Matching(AGs(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align

60. Step(2:(Matching(AGs(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align

61. Step(2:(Matching(AGs(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align

62. Step(2:(Matching(AGs(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align

63. Step(2:(Matching(AGs(
9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align

64. Comprehensive(
XBI(Detection9
• Behavioral XBIs
• Content XBIs
• Structural XBIs

65. Text
Visual
Comprehensive(
XBI(Detection9
Text
Visual
Report9
Generation9
Model(
Generation9
...
Behavior9
Checker9
Element9
Matcher9
Structure9
Checker9
Content9
Checker9
Model(Comparison9
✔9 ✔9
✔9

66. Empirical(Evaluation(
9
Tool: X-PERT (Cross-Platform Error ReporTer)
Research Questions:
• RQ1: Can X-PERT find XBIs in real web
applications?
• RQ2: How does X-PERT compare to the
state-of-art?

67. Experimental(Protocol(
9
v14.0.19 v9.0.99v/s9
Ran(X2PERT(on(Subject(Applications9
Subjects9
12
My Awesome Site
12
My Awesome Site
12
My Awesome Site
+9 =(XBIs9
(Ground(Truth)9Manual(Check9
Error(Reports9 Manual(Check9
=9False(Positives(9
&(Negatives9+9

68. Subjects(
9
Name% Type% States% Transi/ons%
DOM%Nodes%(per%screen)%
max% min% average%
!Organizer! Produc/vity! 13! 99! 27482! 10001! 13051!!
!GrantaBooks! Publisher! 9! 8! 37800! 15625! 25852!!
!DesignTrust! Business! 10! 20! 26437! 7772! 18694!!
!DivineLife! Spiritual! 10! 9! 140611! 9082! 49886!!
!SaiBaba! Religious! 13! 20! 42606! 524! 12162!!
!Breakaway! Sport! 19! 18! 45148! 8191! 13059!!
!Conference! Informa/on! 3! 12! 878! 817! 853!!
!Fisherman! Restaurant! 15! 17! 39146! 15720! 21336!!
!Valleyforge! Lodge! 4! 12! 5416! 4733! 5046!!
!UniMelb! University! 9! 8! 15142! 12131! 13792!!
!Konqueror! SoUware! 5! 4! 17586! 15468! 16187!!
!UBC! Club! 7! 7! 20610! 7834! 12094!!
!BMVBS! Ministry! 5! 20! 19490! 12544! 15695!!
!StarWars! Movie! 10! 9! 28452! 19719! 22626!!!
800(2(140K(elements9
Prior(Art((6)9
Example(and(Survey((4)9
Random((4)9

69. Subjects(
9
Name% Type% States% Transi/ons%
DOM%Nodes%(per%screen)%
max% min% average%
!Organizer! Produc/vity! 13! 99! 27482! 10001! 13051!!
!GrantaBooks! Publisher! 9! 8! 37800! 15625! 25852!!
!DesignTrust! Business! 10! 20! 26437! 7772! 18694!!
!DivineLife! Spiritual! 10! 9! 140611! 9082! 49886!!
!SaiBaba! Religious! 13! 20! 42606! 524! 12162!!
!Breakaway! Sport! 19! 18! 45148! 8191! 13059!!
!Conference! Informa/on! 3! 12! 878! 817! 853!!
!Fisherman! Restaurant! 15! 17! 39146! 15720! 21336!!
!Valleyforge! Lodge! 4! 12! 5416! 4733! 5046!!
!UniMelb! University! 9! 8! 15142! 12131! 13792!!
!Konqueror! SoUware! 5! 4! 17586! 15468! 16187!!
!UBC! Club! 7! 7! 20610! 7834! 12094!!
!BMVBS! Ministry! 5! 20! 19490! 12544! 15695!!
!StarWars! Movie! 10! 9! 28452! 19719! 22626!!!
800(2(140K(elements9

70. Effectiveness(
9
NAME% XBI%
BEHAVIOR% STRUCTURE%
CONTENT%
TOTAL%TEXT% IMAGE%
TP% FP% TP% FP% TP% FP% TP% FP% TP% FP%
Organizer! 10! 1! 0! 9! 0! 0! 0! 0! 0! 10! 0!
GrantaBooks! 27! 16! 0! 11! 0! 0! 0! 0! 0! 27! 0!
DesignTrust! 7! 2! 0! 5! 3! 0! 0! 0! 0! 7! 3!
DivineLife! 11! 7! 0! 3! 6! 1! 0! 0! 0! 11! 6!
SaiBaba! 5! 2! 0! 2! 9! 0! 0! 0! 0! 4! 9!
Breakaway! 13! 0! 0! 10! 2! 0! 0! 0! 0! 10! 2!
Conference! 7! 2! 0! 3! 0! 1! 0! 1! 0! 7! 0!
Fisherman! 5! 1! 0! 3! 1! 0! 1! 1! 0! 5! 2!
Valleyforge! 3! 0! 0! 2! 2! 0! 0! 1! 0! 3! 2!
UniMelb! 2! 2! 0! 0! 0! 0! 0! 0! 1! 2! 1!
Konqueror! 0! 0! 0! 0! 0! 0! 0! 0! 6! 0! 6!
UBC! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!
BMVBS! 1! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!
StarWars! 12! 0! 0! 12! 0! 0! 0! 0! 0! 12! 0!
TOTAL% 103% 33% 0% 60% 23% 2% 1% 3% 7% 98% 31%

71. Effectiveness(
9
NAME% XBI%
BEHAVIOR% STRUCTURE%
CONTENT%
TOTAL%TEXT% IMAGE%
TP% FP% TP% FP% TP% FP% TP% FP% TP% FP%
Organizer! 10! 1! 0! 9! 0! 0! 0! 0! 0! 10! 0!
GrantaBooks! 27! 16! 0! 11! 0! 0! 0! 0! 0! 27! 0!
DesignTrust! 7! 2! 0! 5! 3! 0! 0! 0! 0! 7! 3!
DivineLife! 11! 7! 0! 3! 6! 1! 0! 0! 0! 11! 6!
SaiBaba! 5! 2! 0! 2! 9! 0! 0! 0! 0! 4! 9!
Breakaway! 13! 0! 0! 10! 2! 0! 0! 0! 0! 10! 2!
Conference! 7! 2! 0! 3! 0! 1! 0! 1! 0! 7! 0!
Fisherman! 5! 1! 0! 3! 1! 0! 1! 1! 0! 5! 2!
Valleyforge! 3! 0! 0! 2! 2! 0! 0! 1! 0! 3! 2!
UniMelb! 2! 2! 0! 0! 0! 0! 0! 0! 1! 2! 1!
Konqueror! 0! 0! 0! 0! 0! 0! 0! 0! 6! 0! 6!
UBC! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!
BMVBS! 1! 0! 0! 0! 0! 0! 0! 0! 0! 0! 0!
StarWars! 12! 0! 0! 12! 0! 0! 0! 0! 0! 12! 0!
TOTAL% 103% 33% 0% 60% 23% 2% 1% 3% 7% 98% 31%
RQ1:()
Can(X2PERT(find(XBIs(in(real(web(applications?9
9
Answer:(Yes)

72. Improvement(
9
NAME% XBI%
XQPERT% CROSSCHECK%
Prec% FP% Recl% Dp% Prec% FP% Recl% Dp%
Organizer! 10! 100%! 0! 100%! 0! 80%! 2! 80%! 13!
GrantaBooks! 27! 100%! 0! 100%! 0! 96%! 1! 100%! 0!
DesignTrust! 7! 70%! 3! 100%! 0! 5%! 122! 86%! 3!
DivineLife! 11! 65%! 6! 100%! 0! 29%! 24! 91%! 3!
SaiBaba! 5! 31%! 9! 80%! 0! 7%! 53! 80%! 10!
Breakaway! 13! 83%! 2! 77%! 1! 13%! 49! 54%! 12!
Conference! 7! 100%! 0! 100%! 0! 100%! 0! 100%! 0!
Fisherman! 5! 71%! 2! 100%! 0! 44%! 5! 80%! 8!
Valleyforge! 3! 60%! 2! 100%! 0! 50%! 1! 33%! 0!
UniMelb! 2! 67%! 1! 100%! 0! 7%! 27! 100%! 0!
Konqueror! 0! NA! 6! NA! 0! 0%! 11! NA! 0!
UBC! 0! NA! 0! NA! 0! 0%! 1! NA! 0!
BMVBS! 1! NA! 0! NA! 0! 0%! 2! 0%! 0!
StarWars! 12! 100%! 0! 100%! 0! 10%! 91! 83%! 3!
TOTAL% 103% 77%% 31% 95%% 1% 32%% 389% 74%% 52%

73. Improvement(
9
NAME% XBI%
XQPERT% CROSSCHECK%
Prec% FP% Recl% Dp% Prec% FP% Recl% Dp%
Organizer! 10! 100%! 0! 100%! 0! 80%! 2! 80%! 13!
GrantaBooks! 27! 100%! 0! 100%! 0! 96%! 1! 100%! 0!
DesignTrust! 7! 70%! 3! 100%! 0! 5%! 122! 86%! 3!
DivineLife! 11! 65%! 6! 100%! 0! 29%! 24! 91%! 3!
SaiBaba! 5! 31%! 9! 80%! 0! 7%! 53! 80%! 10!
Breakaway! 13! 83%! 2! 77%! 1! 13%! 49! 54%! 12!
Conference! 7! 100%! 0! 100%! 0! 100%! 0! 100%! 0!
Fisherman! 5! 71%! 2! 100%! 0! 44%! 5! 80%! 8!
Valleyforge! 3! 60%! 2! 100%! 0! 50%! 1! 33%! 0!
UniMelb! 2! 67%! 1! 100%! 0! 7%! 27! 100%! 0!
Konqueror! 0! NA! 6! NA! 0! 0%! 11! NA! 0!
UBC! 0! NA! 0! NA! 0! 0%! 1! NA! 0!
BMVBS! 1! NA! 0! NA! 0! 0%! 2! 0%! 0!
StarWars! 12! 100%! 0! 100%! 0! 10%! 91! 83%! 3!
TOTAL% 103% 77%% 31% 95%% 1% 32%% 389% 74%% 52%
X"PERT)
9
Precision()
((=(77%((
(((((45%!)9
9
Recall()
((=(95%(
(((((14%!)9

74. Improvement(
9
NAME% XBI%
XQPERT% CROSSCHECK%
Prec% FP% Recl% Dp% Prec% FP% Recl% Dp%
Organizer! 10! 100%! 0! 100%! 0! 80%! 2! 80%! 13!
GrantaBooks! 27! 100%! 0! 100%! 0! 96%! 1! 100%! 0!
DesignTrust! 7! 70%! 3! 100%! 0! 5%! 122! 86%! 3!
DivineLife! 11! 65%! 6! 100%! 0! 29%! 24! 91%! 3!
SaiBaba! 5! 31%! 9! 80%! 0! 7%! 53! 80%! 10!
Breakaway! 13! 83%! 2! 77%! 1! 13%! 49! 54%! 12!
Conference! 7! 100%! 0! 100%! 0! 100%! 0! 100%! 0!
Fisherman! 5! 71%! 2! 100%! 0! 44%! 5! 80%! 8!
Valleyforge! 3! 60%! 2! 100%! 0! 50%! 1! 33%! 0!
UniMelb! 2! 67%! 1! 100%! 0! 7%! 27! 100%! 0!
Konqueror! 0! NA! 6! NA! 0! 0%! 11! NA! 0!
UBC! 0! NA! 0! NA! 0! 0%! 1! NA! 0!
BMVBS! 1! NA! 0! NA! 0! 0%! 2! 0%! 0!
StarWars! 12! 100%! 0! 100%! 0! 10%! 91! 83%! 3!
TOTAL% 103% 77%% 31% 95%% 1% 32%% 389% 74%% 52%
X"PERT)
9
Precision()
((=(77%((
(((((45%!)9
9
Recall()
((=(95%(
(((((14%!)9
RQ2:()
How(does(X2PERT(compare(to(the(state2of2art?9
9
Answer:(X"PERT(has(be[er(precision(and(recall)

75. Experimental(Data(&(Tool9
• Release
o Experiments (Crawl Data + Reports)
o Layout testing algorithm implementation
http://gatech.github.io/xpert

76. Related(Work9
o Industrial tools
• Test Suites for Web Browsers – Acid and test262
• MS Expression Web, BrowserShots, BrowserStack
• Browsera, MogoTest, BrowserBite
o Research techniques
• Eaton & Memon [IJWET’07]
• Dallmeier, Burger, Orth & Zeller [JSTools’12]
77

77. Contributions(
9
Empirical(evaluation9
shows(effectiveness9
New(Structural(XBI9
Detection(Algorithm9
body
header
menu
hr1
main
hr2
footer
link1
link2
subHeading
countText
paperList
paper1
paper2
paper3
paper4
button1
button2
above
above
above
above
above
leftOf
above
above
above
left-align
right-align
above
left-align
right-align
above
left-align
right-align
leftOf
top-align
bottom-align
Comprehensive(9
Technique(to(detect(XBIs9
Web$Application.
12
My Awesome Site
Model$Generation. Model$Comparison.
Error$Report.
Study(of(Real(World(XBIs9
XBI$Distribu,on$
Behavior$ 9%$
Content$ Text$ 22%$
Visual$ 30%$
Structure$ 57%$