srawlins/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Performance of new UNUSED_SHOWN_NAME warning

Examining imports, and now the shown names on imports, is an expensive
operation, as it potentially requires examining all of the elements in a
library to discover if an import, or a shown name on an import, is used. So I
wanted to test whether the new UNUSED_SHOWN_NAME warning significantly increases
analysis time.
It obviously should increase analysis time by some amount, as it is doing
extra work that wasn't being done before, but I wanted to measure this increase
nonetheless.
Analysis

I couldn't figure out, with a cursory look, how to use the benchmark_harness
package to test the analyzer package, so I went with a simple time execution.
I tested four different scenarios, all described below. Each scenario involves
one dart file with a thousand import statements that all import a single, second
dart file, showing 2 names per import. A thousand imports is a massive number; I
don't think I've ever seen a real Dart file with so many imports, so this is a
pretty good test.
The test files are generated with the attached mk-unused-shown-names-tests.sh.
Summary

The details are below, but here's the short of it:
In the absolute worst case scenario, which is a dart file with 1000 import lines
that all use "as p", and that each show 2 names. Below the imports, every one of
the first names is used (to avoid UNUSED_IMPORT warnings), but only 990 of the
second names are used, generating 10 UNUSED_SHOWN_NAME warnings. In this
scenario, the existing analyzer took 4.9 seconds to analyze, and the analyzer
with my changes took 5.4 seconds, an increase of 0.5 seconds, or 10%.
10% sounds pretty bad to be honest, but this is the absolute worst case scenario;
that 10% should never translate to whole seconds. In a final test using only
100 imports, where 90 of the second shown names are used, the increased time in
analysis is about 1% (less than 0.1 seconds).
Timing results

1000 imports; each shows 2 names; 1st name always used; only 10 of second name are used


run
current
new


real
user


1
4.821
4.508


2
4.710
4.501


3
4.702
4.490


4
4.700
4.488


5
4.699
4.490


------
-------
-------


avg*
4.704
4.494


Average increase - real time: 0.1s (3%); user time: 0.1s (3%)
1000 imports; each shows 2 names; 1st name always used; only 990 of second name are used


run
current
new


real
user


1
5.320
5.009


2
5.266
4.980


3
5.214
4.941


4
5.258
4.954


5
6.075
5.710


------
-------
-------


avg*
5.246
4.981


Average increase - real time: -0.1s (-2%); user time: -0.1s (-1%)
1000 imports using "as"; each shows 2 names; 1st name always used; only 10 of second name are used


run
current
new


real
user


1
4.766
4.513


2
4.778
4.525


3
4.734
4.486


4
4.794
4.537


5
4.791
4.530


------
-------
-------


avg*
4.778
4.523


Average increase - real time: 0.3s (9%); user time: 0.3s (8%)
1000 imports using "as"; each shows 2 names; 1st name always used; only 990 of second name are used


run
current
new


real
user


1
5.130
4.852


2
4.898
4.642


3
4.973
4.710


4
4.969
4.719


5
4.897
4.677


------
-------
-------


avg*
4.921
4.702


Average increase - real time: 0.5s (10%); user time: 0.5s (10%)
100 imports using "as"; each shows 2 names; 1st name always used; only 90 of second name are used


run
current
new


real
user


1
3.554
3.380


2
3.536
3.352


3
3.544
3.369


4
3.524
3.354


5
3.661
3.486


------
-------
-------


avg*
3.545
3.368


Average increase - real time: 0.0s (1%); user time: 0.0s (1%)
* average of middle 3

  
## mk-unused-shown-names-tests.sh
# Many imports; 2 shows per import; few shown used
###############################################################################

file1=many-imports-show__use-few.dart
file2=many-imports-show__use-few2.dart
imports=1000
uses=10

echo '' > $file1
echo '' > $file2

for i in `seq 1 $imports`; do
  echo "import '$file2' show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
done

# Must use all the A's to avoid UNUSED_IMPORT warnings
for i in `seq 1 $imports`; do
  echo "A`printf %03d $i` a`printf %03d $i`;" >> $file1
done

# Just a few uses of B's
for i in `seq 1 $uses`; do
  echo "B`printf %03d $i` b`printf %03d $i`;" >> $file1
done

for i in `seq 1 $imports`; do
  echo "class A`printf %03d $i` {}" >> $file2
  echo "class B`printf %03d $i` {}" >> $file2
done

# Many imports; 2 shows per import; many shown used
###############################################################################

file1=many-imports-show__use-many.dart
file2=many-imports-show__use-many2.dart
imports=1000
uses=990

echo '' > $file1
echo '' > $file2

for i in `seq 1 $imports`; do
  echo "import '$file2' show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
done

# Must use all the A's to avoid UNUSED_IMPORT warnings
for i in `seq 1 $imports`; do
  echo "A`printf %03d $i` a`printf %03d $i`;" >> $file1
done

# Just a few uses of B's
for i in `seq 1 $uses`; do
  echo "B`printf %03d $i` b`printf %03d $i`;" >> $file1
done

for i in `seq 1 $imports`; do
  echo "class A`printf %03d $i` {}" >> $file2
  echo "class B`printf %03d $i` {}" >> $file2
done


# Many imports; as; 2 shows per import; few shown used
###############################################################################

file1=many-imports-as-show__use-few.dart
file2=many-imports-as-show__use-few2.dart
imports=1000
uses=10

echo '' > $file1
echo '' > $file2

for i in `seq 1 $imports`; do
  echo "import '$file2' as f show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
done

# Must use all the A's to avoid UNUSED_IMPORT warnings
for i in `seq 1 $imports`; do
  echo "f.A`printf %03d $i` a`printf %03d $i`;" >> $file1
done

# Just a few uses of B's
for i in `seq 1 $uses`; do
  echo "f.B`printf %03d $i` b`printf %03d $i`;" >> $file1
done

for i in `seq 1 $imports`; do
  echo "class A`printf %03d $i` {}" >> $file2
  echo "class B`printf %03d $i` {}" >> $file2
done


# Many imports; as; 2 shows per import; many shown used
###############################################################################

file1=many-imports-as-show__use-many.dart
file2=many-imports-as-show__use-many2.dart
imports=1000
uses=990

echo '' > $file1
echo '' > $file2

for i in `seq 1 $imports`; do
  echo "import '$file2' as f show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
done

# Must use all the A's to avoid UNUSED_IMPORT warnings
for i in `seq 1 $imports`; do
  echo "f.A`printf %03d $i` a`printf %03d $i`;" >> $file1
done

# Just a few uses of B's
for i in `seq 1 $uses`; do
  echo "f.B`printf %03d $i` b`printf %03d $i`;" >> $file1
done

for i in `seq 1 $imports`; do
  echo "class A`printf %03d $i` {}" >> $file2
  echo "class B`printf %03d $i` {}" >> $file2
done


# Few imports; as; 2 shows per import; most shown used
###############################################################################

file1=few-imports-as-show__use-most.dart
file2=few-imports-as-show__use-most2.dart
imports=100
uses=90

echo '' > $file1
echo '' > $file2

for i in `seq 1 $imports`; do
  echo "import '$file2' as f show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
done

# Must use all the A's to avoid UNUSED_IMPORT warnings
for i in `seq 1 $imports`; do
  echo "f.A`printf %03d $i` a`printf %03d $i`;" >> $file1
done

# Just a few uses of B's
for i in `seq 1 $uses`; do
  echo "f.B`printf %03d $i` b`printf %03d $i`;" >> $file1
done

for i in `seq 1 $imports`; do
  echo "class A`printf %03d $i` {}" >> $file2
  echo "class B`printf %03d $i` {}" >> $file2
done
run	current	new
	real	user
1	4.821	4.508
2	4.710	4.501
3	4.702	4.490
4	4.700	4.488
5	4.699	4.490
------	-------	-------
avg*	4.704	4.494
run	current	new
	real	user
1	5.320	5.009
2	5.266	4.980
3	5.214	4.941
4	5.258	4.954
5	6.075	5.710
------	-------	-------
avg*	5.246	4.981
run	current	new
	real	user
1	4.766	4.513
2	4.778	4.525
3	4.734	4.486
4	4.794	4.537
5	4.791	4.530
------	-------	-------
avg*	4.778	4.523
run	current	new
	real	user
1	5.130	4.852
2	4.898	4.642
3	4.973	4.710
4	4.969	4.719
5	4.897	4.677
------	-------	-------
avg*	4.921	4.702
run	current	new
	real	user
1	3.554	3.380
2	3.536	3.352
3	3.544	3.369
4	3.524	3.354
5	3.661	3.486
------	-------	-------
avg*	3.545	3.368
	# Many imports; 2 shows per import; few shown used
	###############################################################################

	file1=many-imports-show__use-few.dart
	file2=many-imports-show__use-few2.dart
	imports=1000
	uses=10

	echo '' > $file1
	echo '' > $file2

	for i in `seq 1 $imports`; do
	echo "import '$file2' show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
	done

	# Must use all the A's to avoid UNUSED_IMPORT warnings
	for i in `seq 1 $imports`; do
	echo "A`printf %03d $i` a`printf %03d $i`;" >> $file1
	done

	# Just a few uses of B's
	for i in `seq 1 $uses`; do
	echo "B`printf %03d $i` b`printf %03d $i`;" >> $file1
	done

	for i in `seq 1 $imports`; do
	echo "class A`printf %03d $i` {}" >> $file2
	echo "class B`printf %03d $i` {}" >> $file2
	done

	# Many imports; 2 shows per import; many shown used
	###############################################################################

	file1=many-imports-show__use-many.dart
	file2=many-imports-show__use-many2.dart
	imports=1000
	uses=990

	echo '' > $file1
	echo '' > $file2

	for i in `seq 1 $imports`; do
	echo "import '$file2' show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
	done

	# Must use all the A's to avoid UNUSED_IMPORT warnings
	for i in `seq 1 $imports`; do
	echo "A`printf %03d $i` a`printf %03d $i`;" >> $file1
	done

	# Just a few uses of B's
	for i in `seq 1 $uses`; do
	echo "B`printf %03d $i` b`printf %03d $i`;" >> $file1
	done

	for i in `seq 1 $imports`; do
	echo "class A`printf %03d $i` {}" >> $file2
	echo "class B`printf %03d $i` {}" >> $file2
	done


	# Many imports; as; 2 shows per import; few shown used
	###############################################################################

	file1=many-imports-as-show__use-few.dart
	file2=many-imports-as-show__use-few2.dart
	imports=1000
	uses=10

	echo '' > $file1
	echo '' > $file2

	for i in `seq 1 $imports`; do
	echo "import '$file2' as f show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
	done

	# Must use all the A's to avoid UNUSED_IMPORT warnings
	for i in `seq 1 $imports`; do
	echo "f.A`printf %03d $i` a`printf %03d $i`;" >> $file1
	done

	# Just a few uses of B's
	for i in `seq 1 $uses`; do
	echo "f.B`printf %03d $i` b`printf %03d $i`;" >> $file1
	done

	for i in `seq 1 $imports`; do
	echo "class A`printf %03d $i` {}" >> $file2
	echo "class B`printf %03d $i` {}" >> $file2
	done


	# Many imports; as; 2 shows per import; many shown used
	###############################################################################

	file1=many-imports-as-show__use-many.dart
	file2=many-imports-as-show__use-many2.dart
	imports=1000
	uses=990

	echo '' > $file1
	echo '' > $file2

	for i in `seq 1 $imports`; do
	echo "import '$file2' as f show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
	done

	# Must use all the A's to avoid UNUSED_IMPORT warnings
	for i in `seq 1 $imports`; do
	echo "f.A`printf %03d $i` a`printf %03d $i`;" >> $file1
	done

	# Just a few uses of B's
	for i in `seq 1 $uses`; do
	echo "f.B`printf %03d $i` b`printf %03d $i`;" >> $file1
	done

	for i in `seq 1 $imports`; do
	echo "class A`printf %03d $i` {}" >> $file2
	echo "class B`printf %03d $i` {}" >> $file2
	done


	# Few imports; as; 2 shows per import; most shown used
	###############################################################################

	file1=few-imports-as-show__use-most.dart
	file2=few-imports-as-show__use-most2.dart
	imports=100
	uses=90

	echo '' > $file1
	echo '' > $file2

	for i in `seq 1 $imports`; do
	echo "import '$file2' as f show A`printf %03d $i`, B`printf %03d $i`;" >> $file1
	done

	# Must use all the A's to avoid UNUSED_IMPORT warnings
	for i in `seq 1 $imports`; do
	echo "f.A`printf %03d $i` a`printf %03d $i`;" >> $file1
	done

	# Just a few uses of B's
	for i in `seq 1 $uses`; do
	echo "f.B`printf %03d $i` b`printf %03d $i`;" >> $file1
	done

	for i in `seq 1 $imports`; do
	echo "class A`printf %03d $i` {}" >> $file2
	echo "class B`printf %03d $i` {}" >> $file2
	done