Skip to content

Instantly share code, notes, and snippets.

@geekodour

geekodour/research.md Secret

Created April 19, 2023 08:07
Show Gist options
  • Save geekodour/69a0c56763a3e4262e8830e4dc3eb7cd to your computer and use it in GitHub Desktop.
Save geekodour/69a0c56763a3e4262e8830e4dc3eb7cd to your computer and use it in GitHub Desktop.
Download ZIP
Raw
research.md

Citations and References

A citation is a mention of an external source in the body of a paper ('in-text' or 'inline'). This is a very common citation style: [1] This is another: (Kraiser, 2011). However, there are many citation styles, and for a published paper, its citation style is usually dictated by the publisher.

A reference is a more full textual description of an external source, designed to allow the reader to obtain the referenced item as a primary source. References are typically collected in a list in a References or Bibliography section toward the end of the paper. Usually, every reference must have been cited somewhere in the paper, and usually every citation has a corresponding reference. However, there are many reference styles, and, for example in APA Style, cited personal correspondence that was private and is not accessible to the reader does not need a corresponding reference. The reference style is typically defined as part of the citation style.

The Citation Graph

Whatever their format, citations and/or references act as edges in a vast knowledge graph, known as The Citation Graph. In this context, work A is said to cite work B, if work A contains a citation or reference to paper B. The citation graph is a directed graph, since the act of work A citing work B does not imply that work B cites work A. i.e. citation is an asymmetric relation between works.

The citation graph is for the most part acyclic - most papers only cite papers published before them. But some publication processes take time and allow a paper to be updated between being accepted and finally published (the so-called 'camera ready' version), allowing a published paper to cite papers published in parallel, and even for two papers to cite one another. This is rare, and usually not an issue unless you are writing algorithms to do analysis on the citation graph.

The Citation Graph has been called "one of humankind's most important intellectual achievements" (by Dario Taraborelli (@readermeter), the Director of Research at the @Wikimedia Foundation).

Citation Graph Visualization

There are a number of free tools that can visualize portions of the citation graph in different ways:

They are, however, in various states of disrepair, and generally have the feel of experiments. While we share the author's disappointment that there are no better visual tools to explore and discover connections between publications, it is perfectly possible to mine the citation graph without the help of such tools. This undoubtedly explains why there are no tools (at least free tools) that have made it past this stage.

That said, if you feel like working on such a tool, please contact Jon Currey to discuss possible features.

Research Discovery

Research Paper Search and Recommendation Sites

There are a number of websites which index, cross-reference and rank research papers. To get the most out of them, create an account and star/like/follow etc. people and papers you have found insightful or helpful. Then the site will recommend related work to you, often with uncanny accuracy!

Some of the best:

Even without creating an acccount, Google Scholar has some very useful features. Click on the title of a paper and look for a link of the form "Cited by nnn" where nnn is the number of papers that cite this paper. Clicking on that link will bring up links to each of those papers. You can sort the papers by the date of their publication, or by Google's assessment of their relevance. You can also search just within these publications, to further narrow your search.

Curated Research Discovery and Discussion

There are many academics, engineers and other interested individuals who publish personal selections of research that are relevant to their specialism or interests. Some of the ones HashiCorp consults most frequently are:

You can also find lists of papers relevant to a topic by doing a web search for the topic (or combinations of key words releated to the topic) and adding "bibliograpy", "reading list" or "survey" to the end. Many of these bibliographies and surverys were themselves published via peer-reviewed publication, which means multiple reviewers should have reviewed them and decided they are a decent representation of work in that space.

There is even a website that curates lists of bibliopgraphies by research topic.

Associations and Professional Bodies

All of them sponsor and/or run a number of conferences on different areas of computer science.

Conference Tracking Sites

There are many sites that use different methodologies to rank academic conferences (often based on some measure of 'impact'), including computer science conferences. For example:

Other sites focus on tracking the deadlines for submission to upcoming conferences. For example:

Any of these can be used to discover conferences you were not previously aware of.

Open Access

Digitization has lowered the material cost of distributing academic papers, just has it has other forms of media. There is an active debate going on about the extent to which organizations can and should be allowed to charge for access to content. Conferences and the review process still have high costs. Should this be subsizided by charging for access to papers, or by other means? HashiCorp is committed to open access. Please educate yourself and make an informed decision regarding paying for access to academic content.

Reading and Evaluating Research

How to Read a Paper

There are many good guides on how to read academic papers. You will develop your own method, but each of them can give you good suggestions to try. Some good places to start:

Advice on how to write a good paper also gives you insight into how to read (and evaluate) a paper:

Known Dangers and Biases (aka Here Be Dragons)

Places to Discuss Research

In Real Life

On-Line

Keeping Track of It All

Techniques

Tools

Tools are very personal, and actually not the key issue. However, some tools offer some nice features.

  • Pen and paper
    • The act of writing and/or drawing is widely reported to improve information retention.
    • Never has an outage (unless you spill your drink on it!)
  • Google Docs
    • Collaborative editing
    • Search within your Google Drive account
  • Evernote
    • Web clippers from Chrome and Firefox. Will clip PDFs as well as web pages.
    • Premium vertsion OCRs all images (including whole pages in scanned PDFs)
    • Indexes everything. Shows top 3 related items from your personal corpus each time you clip something.
  • Papers app
  • https://orgmode.org/
  • Jupyter Notebooks (formerly IPython Notebooks)
    • No longer restricted to Python, these allow you to embed experiments with notes. An increasing number of cloud providers offer them for free. Or you can host your own.
    • JupyterHub For setting up your own installation in a public cloud or elsewhere.
    • Azure Notebooks
    • Google Colaboratory

HashiCorp's Experience Applying Research to Production

Backlog

A backlog is a great way to decompose a large, daunting project into small, achievable tasks. This technique can be applied to the discovery and evaluation of academic research. We will use one for this resource too.

The current backlog of resources to evaluate and potentiall integrate into this document:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment