Skip to content

Instantly share code, notes, and snippets.

@cat-state

cat-state/optical-nn.md

Created September 8, 2022 18:56
Show Gist options
  • Save cat-state/7b827d424f6636a1ac111a6e786f4ad1 to your computer and use it in GitHub Desktop.
Save cat-state/7b827d424f6636a1ac111a6e786f4ad1 to your computer and use it in GitHub Desktop.
Download ZIP
optical nn
Raw
optical-nn.md

optical-nn

This repository for organising information as pertaining to the DIY construction of a non-linear optical neural network.

Module

          +------------SINGLE LAYER+--------------------------+
          |                                                   |
          |     +-----+ +--------------+ +-----+ +-------+    |
       +----->  | W1  | |     NON      | | W2  | | μLENS |    |
INPUT  |----->  | SLM | |     LIN      | | SLM | |       |    |
       +----->  |     | |              | |     | |       |    |
          |     +-----+ +--------------+ +-----+ +-------+    |
          |                                                   |
          |                                                   |
          +---------------------------------------------------+

How it works:

  1. Input light is given a spatial polarization pattern before input to the network
  2. The W1 SLM (spatial light modulator) is made by removing the polarizer and backlight from an LCD. This means each "pixel" in it performs a weighted sum of the input light, weighted by the correlation between the polarization of the cell & the polarization of any region of the input light.
  3. The nonlinearity transforms the light...non-linearly. Different components could be used to implement this - a matrix of photodiodes followed by a matrix of LEDs, a camera sensor and small display, a nonlinear optical material such as dye-doped liquid crystal.
  4. The W2 SLM (constructed same as W1 SLM, but with a different pattern) imparts a spatially varying polarization pattern onto the output light, for consumption by the next layer.
  5. The microlens array re-scatters the output light from the W2 SLM so that it can reach all of the next layer's input SLM. By selecting a longer focal-distance we could also make the light from each cell only reach a subset of the cells of the next layer, to work around the low dynamic range of the polarization pattern based routing.

To learn a model, we first figure out the transfer function of our non-linearity and make a suitable simulator. We make a Pytorch (or whatever framework) layer that simulates the filtering by polarization.

Forseen problems

  • The polarization pattern based routing probably has low dynamic range
  • The optical non-linearity might also only have a low dynamic range
  • Perhaps would need to amplify the signal after the non-linearity.
  • How to simulate various losses in the system?
  • Optimising the device directly (i.e, no simulator) would be slow

Random notes

  • Using an RGB LCD as the base for the SLMs could give 3x more neurons per layer that can route to each other?

Papers

Videos

Choices for non-linearities

  • Matrix of photo-diodes (or reverse biased LEDs) wired up 1-1 with a matrix of LEDs.

    • Pro: can just buy it
    • Pro: easy to amplify input signal
    • Pro: very likely to work??
    • Con: more labor intensive (have to individually wire up photo-diode to LED)

  • "Image amplifier" tube (e.g, for night-vision)

    • Pro: good amplification
    • Pro: very likely to work
    • Pro: can just buy it
    • Con: very expensive, hard to DIY (since it's a vacuum tube)

  • Camera sensor + small screen for output (use analog camera & analog signal for max lazyness)

    • Pro: can just buy it
    • Pro: easy to assemble an integrate
    • Con: maybe expensive to do many layers like this
    • Con: if the output screen is an LCD, it imparts a polarization onto the light based on its intensity, which we'll have to figure out how to deal with

  • Non-linear optical material like dye doped liquid crystals

    • Pro: all optical system! very cool..

    • Pro: no wiring

    • Pro/con: non-linear effect is angle dependent, so additional degree-of-freedom in how to route signals. But could be hard to simulate/measure

    • Con: idk if it will work

    • Con: unknown fabrication method?

    • Con: if the non-linearity is too low then would have to figure out how to amplify the signal (can use the phase-shift imparted onto it to intensity modulate a laser, as in Khoo et al (2000) above)

    • See also: materials of OASLAMs like: https://www.nature.com/articles/lsa201532 - has fabrication instructions! The fabrication seems plausible to do. But the device can only turn UV light into non-UV light. - the OASLAM can work as a nonlinearity by having a "write beam" write a hologram into a substance, which is replicated onto the "read beam". To use as a nonlinearity, the "write beam" is the output of the previous layer and the "read beam" is used as the output of the nonlinearity. However, it seems that the read beam and write beams have to be of different wavelengths for transmissive OASLAMs (they could be the same for reflective ones maybe). - in that article they say that the "write beam" wavelength can be controlled by varying the nanoparticle size. If the nanoparticle size could be constrained enough, you could use alternate wavelengths as you cascade down.

    • see also: ancient technology: https://ieeexplore.ieee.org/document/1478833 - it seems at one time optically addressed "liquid crystal light valves" were used to turn CRTs into projectors. - e.g: http://pro.jvc.com/pro/hjt/technology/download/sid99.pdf

Fabrication

  1. SLMs: Buy cheap transparent or transflective LCD, remove polarizer. Could also buy normal LCD but then would also have to remove backlight & the SLM might be less transparent?

  2. Microlens array: something like https://www.holoor.co.il/broadband-diffuser-2/ but cheaper - just want to evenly spread the output of each pixel into a cone. Keywords: broadband laser homogenizer.

  3. It seems like https://www.eyrise.com/eyrise-frequently-asked-questions/ uses some kind of dye-doped LC for smart windows. In addition you can find lots of polymer-doped LC on Alibaba (for smart windows).

  4. You can buy some here: https://www.beamco.com/Products ?

  5. My guess for how to fabricate the methyl-red doped liquid crystals

  • Buy:
    • Liquid Crystal (5CB seems to be the most used in papers)
    • 50-100 micron glass beads
    • Methyl Red
    • Two thin glass sheets (e.g, slides)
    • Sealant/Gasket
  • Make:
    • Spin-coater (can use an old hard drive, or a quadcopter motor, many youtube videos on this)
  1. Spin coat one glass slide with dye
  2. Mix LC and beads
  3. Put LC on glass slide then put the other slide on top and squeeze together
  4. Seal the sides?
  5. Done
  • Questions:
    • Maybe can just mix the LC with the dye & skip the spin-coating?
    • It seems like most papers use a 0.5% dye by weight proportion w.r.t the amount of LC. How to ensure this concentration when spin-coating the slide?
    • Glass beads can be removed if don't care about exact thickness of the cell?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment