Skip to content

Instantly share code, notes, and snippets.

@paulirwin
Created October 1, 2013 15:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save paulirwin/6780136 to your computer and use it in GitHub Desktop.
Save paulirwin/6780136 to your computer and use it in GitHub Desktop.
Lucene.net 4.3 failing test case. writer.ForceMerge(1) fails. Also fails in the same way if you commit enough docs to automatically merge.
public void RunTest()
{
var analyzer = new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_43);
using (var dir = FSDirectory.Open("C:\\Index\\43TestCases\\TestCase1"))
using (var writer = new IndexWriter(dir, new IndexWriterConfig(Lucene.Net.Util.Version.LUCENE_43, analyzer)))
{
var doc = new Document();
doc.Add(new IntField("DocID", 123, Field.Store.YES));
doc.Add(new TextField("Text", "Ramaria botrytis, commonly known as the clustered coral, the pink-tipped coral mushroom, or the cauliflower coral, is an edible species of coral fungus in the family Gomphaceae. Its robust fruit body can grow up to 15 cm (6 in) in diameter and 20 cm (8 in) tall, and resembles some marine coral. Its dense branches, which originate from a stout, massive base, are swollen at the tips and divided into several small branchlets. The branches are initially whitish but age to buff or tan, with tips that are pink to reddish. The flesh is thick and white. The spores, yellowish in deposit, are ellipsoid, feature longitudinal striations, and measure about 13.8 by 4.7 micrometers.", Field.Store.YES));
writer.AddDocument(doc);
writer.Commit();
doc = new Document();
doc.Add(new IntField("DocID", 234, Field.Store.YES));
doc.Add(new TextField("Text", "The type species of the genus Ramaria, R. botrytis was first described scientifically in 1797 by mycologist Christiaan Hendrik Persoon. A widely distributed species, it is found in North America, North Africa, central and eastern Europe, Australia, and Asia. The fungus is mycorrhizal with broadleaf trees, and fruits on the ground in wooded areas. There are several species of coral fungi that are superficially similar in appearance to R. botrytis, and although comparison of habitat or characteristics like color or branching morphology is often sufficient for identification, sometimes microscopy is required to definitively distinguish between them. Fruit bodies of Ramaria botrytis are edible, and young specimens have a mild, fruity taste. Some authors warn of laxative effects in susceptible individuals. The fungus contains several chemical compounds with in vitro biological activity, and fruit bodies have antimicrobial activity against several species and strains of drug-resistant bacteria that cause disease in humans.", Field.Store.YES));
writer.AddDocument(doc);
writer.Commit();
doc = new Document();
doc.Add(new IntField("DocID", 345, Field.Store.YES));
doc.Add(new TextField("Text", "The species was first named as Clavaria botrytis in 1797 by Christian Hendrik Persoon.[2] In 1821, Elias Magnus Fries sanctioned the genus name Clavaria, and treated Ramaria as a section of Clavaria.[3] It was given its current name in 1918 by Adalbert Ricken.[4] Obsolete historical synonyms include Gotthold Hahn's 1883 Corallium botrytis[5] and A. A. Pearson's variety Clavaria botrytis var. alba,[6] which is no longer recognized as an independent taxon.[1] Currie Marr and Daniel Stuntz described the variety R. botrytis var. aurantiiramosa in their 1973 monograph of western Washington Ramaria;[7] Edwin Schild and G. Ricci described variety compactospora from Italy in 1998.[8] In 1950, E.J.H. Corner published George F. Atkinson's 1908 Clavaria holorubella as R. botrytis var. holorubella,[9] but this taxon is now known as the independent species Ramaria holorubella.", Field.Store.YES));
writer.AddDocument(doc);
writer.Commit();
doc = new Document();
doc.Add(new IntField("DocID", 4567890, Field.Store.YES));
doc.Add(new TextField("Text", "Ramaria botrytis was designated the type species of Ramaria in 1933 by Marinus Anton Donk.[16] Modern molecular analysis indicates that Ramaria is a polyphyletic assemblage of species with clavarioid fruit bodies.[17][18] According to the infrageneric classification scheme proposed by Marr and Stuntz, R. botrytis is included in the subgenus Ramaria, which includes species that have grooved spores, clamps present in the hyphae, and fruit bodies with a large, profusely branched cauliflower-like appearance.[7] Phylogenetic analysis of nuclear large subunit ribosomal DNA suggests that R. botrytis is closely related to R. rubripermanens and R. rubrievanescens, and that these species form a clade that is sister (sharing a recent common ancestor) to the false truffle genus Gautieria, the most derived group within the studied taxa.", Field.Store.YES));
writer.AddDocument(doc);
writer.Commit();
doc = new Document();
doc.Add(new IntField("DocID", 100000, Field.Store.YES));
doc.Add(new TextField("Text", "The fruit bodies produced by the fungus are 7 to 15 cm (2.8 to 5.9 in) wide and 6 to 20 cm (2.4 to 7.9 in) tall.[9] They are fleshy cauliflower-like masses with a stout central stem that splits into a few lower primary branches before branching densely above. The stem is short and thick—between 1.5 and 6 cm (0.6 and 2.4 in) in diameter—and tapers downward. Initially white, in age both the stem and branches turn pale yellow to buff to tan.[13] Old fruit bodies can fade to become almost white,[19] or may be ochre due to fallen spores.[14] The branching pattern is irregular, with the primary branches few and thick—typically 2–3 cm (0.8–1.2 in)—and the final branches slender (2–3 mm),[13] and usually terminated with five to seven branchlets.[7] The branchlet tips are pink to purplish-red. The flesh is solid and white,[13] and has an odor described variously as indistinct[20] or pleasant.[21] A drop of Melzer's reagent applied to the stem tissue reveals a weak amyloid staining reaction that often requires more than 30 minutes to develop. This reaction can be used to help distinguish R. botrytis from other similar fungi.[7]", Field.Store.YES));
writer.AddDocument(doc);
writer.Commit();
doc = new Document();
doc.Add(new IntField("DocID", 234567890, Field.Store.YES));
doc.Add(new TextField("Text", "Spores are produced by basidia on the outer surface of the branches. Viewed in deposit, the spores are pale yellow. Microscopically, they have fine longitudinal or oblique striations that often fuse together in a vein-like network. They range in shape from roughly cylindrical to sigmoid (curved like the letter \"S\"), and their dimensions are 12–16 by 4–5 µm.[22][23] Basidia are four-spored (occasionally two-spored), and measure 59–82 by 8–11 µm. The sterigmata (slender projections of the basidia that attach to the spores) are 4–8 µm long. The hymenium and subhymenium (the tissue layer immediately under the hymenium) combined are about 80 µm thick. Hyphae comprising the subhymenium are interwoven, 2.5–4.5 µm in diameter, thin-walled, and clamped.[7]", Field.Store.YES));
writer.AddDocument(doc);
writer.Commit();
writer.ForceMerge(1);
writer.Commit();
}
}
@paulirwin
Copy link
Author

Exception is an IndexOutOfRangeException in Lucene.Net.Codecs.Compressing.LZ4, line 54. It's trying to access a byte out of a byte buffer of length 18432 at index 134217728. You must enable breaking on thrown IndexOutOfRangeExceptions to easily catch it.

@paulirwin
Copy link
Author

Found order of operations issue in Packed64. Fixed here: paulirwin/lucene.net@279224a

@paulirwin
Copy link
Author

Code above still gives exception, now it's the "already finished" exception in FST.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment