Skip to content

Instantly share code, notes, and snippets.

@HugoDF

HugoDF/LICENSE.md

Last active Jul 25, 2020
Embed
What would you like to do?
Integrate lunrjs with a Hugo (gohugo.io) site.
const fs = require('fs').promises;
const {promisify} = require('util');
const frontMatterParser = require('parser-front-matter');
const parse = promisify(frontMatterParser.parse.bind(frontMatterParser));
async function loadPostsWithFrontMatter(postsDirectoryPath) {
const postNames = await fs.readdir(postsDirectoryPath);
const posts = await Promise.all(
postNames.map(async fileName => {
const fileContent = await fs.readFile(
`${postsDirectoryPath}/${fileName}`,
'utf8'
);
const {content, data} = await parse(fileContent);
return {
content: content.slice(0, 3000),
...data
};
})
);
return posts;
}
const lunrjs = require('lunr');
function makeIndex(posts) {
return lunrjs(function() {
this.ref('title');
this.field('title');
this.field('content');
this.field('tags');
posts.forEach(p => {
this.add(p);
});
});
}
async function run() {
const posts = await loadPostsWithFrontMatter(`${__dirname}/content/post`);
const index = makeIndex(posts);
console.log(JSON.stringify(index));
}
run()
.then(() => process.exit(0))
.catch(error => {
console.error(error.stack);
process.exit(1);
});
node ./build-lunrjs-index.js > static/search-index.json

The MIT License (MIT) Copyright (c) 2019-2020 Hugo Di Francesco

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

{
"devDependencies": {
"lunr": "^2.3.6",
"parser-front-matter": "^1.6.4"
},
"license": "MIT"
}
<form method="get" action="">
<input id="search" name="q" type="text" />
<button type="submit" class="button">Search</button>
<a href="/search">Clear</a>
</form>
<div id="#app"></div>
<script src="https://unpkg.com/lunr/lunr.js"></script>
<!-- Generate a list of posts so we can display them -->
{{ $p := slice }}
{{ range (where .Site.RegularPages "Section" "==" "post") }}
{{ $post := dict "link" .RelPermalink "title" .Title "content" (substr .Plain 0 200) -}}
{{ $p = $p | append $post -}}
{{ end }}
<script>
const posts = JSON.parse(
{{ $p | jsonify }}
);
const query = new URLSearchParams(window.location.search);
const searchString = query.get('q');
document.querySelector('#search').value = searchString;
const $target = document.querySelector('#app');
// Our index uses title as a reference
const postsByTitle = posts.reduce((acc, curr) => {
acc[curr.title] = curr;
return acc;
}, {});
fetch('/gen/search-index.json').then(function (res) {
return res.json();
}).then(function (data) {
const index = lunr.Index.load(data);
const matches = index.search(searchString);
const matchPosts = [];
matches.forEach((m) => {
matchPosts.push(postsByTitle[m.ref]);
});
if (matchPosts.length > 0) {
$target.innerHTML = matchPosts.map(p => {
return `<div>
<h3><a href="${p.link}">${p.title}</a></h3>
<p>${p.content}...</p>
</div>`;
}).join('');
} else {
$target.innerHTML = `<div>No search results found</div>`;
}
});
@psharma04

This comment has been minimized.

Copy link

@psharma04 psharma04 commented Aug 7, 2019

Lines 10-14 are displaying as text, not doing anything.

@HugoDF

This comment has been minimized.

Copy link
Owner Author

@HugoDF HugoDF commented Aug 8, 2019

@psharma04 the HTML only works as a Hugo template

@oonid

This comment has been minimized.

Copy link

@oonid oonid commented Aug 11, 2019

@HugoDF do you have any example how to integrate search.html to Hugo template? or may be another article at codewithhugo? thank you.

@HugoDF

This comment has been minimized.

Copy link
Owner Author

@HugoDF HugoDF commented Aug 11, 2019

The post is at https://codewithhugo.com/hugo-lunrjs-search-index/

The build-lunrjs-index.js script needs to be built before the corresponding Hugo template is used to generate the site.

@exprez135

This comment has been minimized.

Copy link

@exprez135 exprez135 commented Nov 7, 2019

I'm trying to implement this in a site, but am getting no search results. Firefox's Console shows this error each time I attempt to search: "TypeError: $target is null".

Edit: Also, if there are any folders in the content/post/ directory (e.g. if I have subdirectories to split by year and month), the index script ends with the error "Error: EISDIR: illegal operation on a directory, read". Any way to recursively index all directories?

@HugoDF

This comment has been minimized.

Copy link
Owner Author

@HugoDF HugoDF commented Nov 7, 2019

@exprez135 for recursive search, you'll have to implement that, get all the paths and then load all the posts into memory, I would change https://gist.github.com/HugoDF/aac2e529f79cf90d2050d7183571684b#file-build-lunrjs-index-js-L8-L9 onwards

The only situation where $target would be null is if you don't have <div id="app"></div> somewhere in your HTML template https://gist.github.com/HugoDF/aac2e529f79cf90d2050d7183571684b#file-search-html-L6

@exprez135

This comment has been minimized.

Copy link

@exprez135 exprez135 commented Apr 17, 2020

By the way, I successfully implemented that recursive search in my project using readdirp in place of readdir and some other work-arounds. You can find it at https://git.sr.ht/~exprez135/taliaferro/tree/master/scripts/build-lunrjs-index.js. My search template is at https://git.sr.ht/~exprez135/mediumish-taliaferro/tree/master/layouts/search-page/search.html.

And I just realized that I used your code without a license. Is there any way you could specify a license for people to use or declare it in the public domain?

Thank you for writing this!

@HugoDF

This comment has been minimized.

Copy link
Owner Author

@HugoDF HugoDF commented Apr 17, 2020

@exprez135 added MIT license + set it as the license in package.json 😄 thanks for the nudge I'm glad it helped you.

@ttgiang

This comment has been minimized.

Copy link

@ttgiang ttgiang commented Jul 14, 2020

Appreciate the code. Learning a lot from it. Would you mind sharing a sample of the data file. I'm new to this and am trying to make it work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.