Skip to content

Instantly share code, notes, and snippets.

@slava-vishnyakov
Last active November 4, 2024 09:03
Show Gist options
  • Save slava-vishnyakov/16076dff1a77ddaca93c4bccd4ec4521 to your computer and use it in GitHub Desktop.
Save slava-vishnyakov/16076dff1a77ddaca93c4bccd4ec4521 to your computer and use it in GitHub Desktop.
How to upload images with TipTap editor
  1. Create a file Image.js from the source below (it is almost a copy of Image.js from tiptap-extensions except that it has a constructor that accepts uploadFunc (function to be called with image being uploaded) and additional logic if(upload) { ... } else { ... previous base64 logic .. } in the new Plugin section.
import {Node, Plugin} from 'tiptap'
import {nodeInputRule} from 'tiptap-commands'

/**
 * Matches following attributes in Markdown-typed image: [, alt, src, title]
 *
 * Example:
 * ![Lorem](image.jpg) -> [, "Lorem", "image.jpg"]
 * ![](image.jpg "Ipsum") -> [, "", "image.jpg", "Ipsum"]
 * ![Lorem](image.jpg "Ipsum") -> [, "Lorem", "image.jpg", "Ipsum"]
 */
const IMAGE_INPUT_REGEX = /!\[(.+|:?)\]\((\S+)(?:(?:\s+)["'](\S+)["'])?\)/;

export default class Image extends Node {

    constructor(name, parent, uploadFunc = null) {
        super(name, parent);
        this.uploadFunc = uploadFunc;
    }

    get name() {
        return 'image'
    }

    get schema() {
        return {
            inline: true,
            attrs: {
                src: {},
                alt: {
                    default: null,
                },
                title: {
                    default: null,
                },
            },
            group: 'inline',
            draggable: true,
            parseDOM: [
                {
                    tag: 'img[src]',
                    getAttrs: dom => ({
                        src: dom.getAttribute('src'),
                        title: dom.getAttribute('title'),
                        alt: dom.getAttribute('alt'),
                    }),
                },
            ],
            toDOM: node => ['img', node.attrs],
        }
    }

    commands({ type }) {
        return attrs => (state, dispatch) => {
            const { selection } = state;
            const position = selection.$cursor ? selection.$cursor.pos : selection.$to.pos;
            const node = type.create(attrs);
            const transaction = state.tr.insert(position, node);
            dispatch(transaction)
        }
    }

    inputRules({ type }) {
        return [
            nodeInputRule(IMAGE_INPUT_REGEX, type, match => {
                const [, alt, src, title] = match;
                return {
                    src,
                    alt,
                    title,
                }
            }),
        ]
    }

    get plugins() {
        const upload = this.uploadFunc;
        return [
            new Plugin({
                props: {
                    handleDOMEvents: {
                        drop(view, event) {
                            const hasFiles = event.dataTransfer
                                && event.dataTransfer.files
                                && event.dataTransfer.files.length;

                            if (!hasFiles) {
                                return
                            }

                            const images = Array
                                .from(event.dataTransfer.files)
                                .filter(file => (/image/i).test(file.type));

                            if (images.length === 0) {
                                return
                            }

                            event.preventDefault();

                            const { schema } = view.state;
                            const coordinates = view.posAtCoords({ left: event.clientX, top: event.clientY });

                            images.forEach(async image => {
                                const reader = new FileReader();

                                if(upload) {
                                    const node = schema.nodes.image.create({
                                        src: await upload(image),
                                    });
                                    const transaction = view.state.tr.insert(coordinates.pos, node);
                                    view.dispatch(transaction)
                                } else {
                                    reader.onload = readerEvent => {
                                        const node = schema.nodes.image.create({
                                            src: readerEvent.target.result,
                                        });
                                        const transaction = view.state.tr.insert(coordinates.pos, node);
                                        view.dispatch(transaction)
                                    };
                                    reader.readAsDataURL(image)
                                }
                            })
                        },
                    },
                },
            }),
        ]
    }

}
  1. Import it:
import Image from './Image';

async function upload(file) {
...
}

new Editor({
  extensions: [
    ...
    new Image(null, null, upload),
    ...
  1. Implement the upload function:
async function upload(file) {
  let formData = new FormData();
  formData.append('file', file);
  const headers = {'Content-Type': 'multipart/form-data'};
  const response = await axios.post('/upload', formData, {headers: headers} );
  return response.data.src;
},

This POSTs using axios to /upload and expects a JSON back of this form:

{"src": "https://yoursite.com/images/uploadedimage.jpg"}
  1. Implement server-side logic for /upload

  2. If you want to support pasting of images, modify Image.js starting at props:, ending at handleDOMEvents (re-factor common parts if you want to)

props: {
    handlePaste(view, event, slice) {
        const items = (event.clipboardData  || event.originalEvent.clipboardData).items;
        for (const item of items) {
            if (item.type.indexOf("image") === 0) {
                event.preventDefault();
                const { schema } = view.state;

                const image = item.getAsFile();

                if(upload) {
                    upload(image).then(src => {
                        const node = schema.nodes.image.create({
                            src: src,
                        });
                        const transaction = view.state.tr.replaceSelectionWith(node);
                        view.dispatch(transaction)
                    });

                } else {
                    const reader = new FileReader();
                    reader.onload = readerEvent => {
                        const node = schema.nodes.image.create({
                            src: readerEvent.target.result,
                        });
                        const transaction = view.state.tr.replaceSelectionWith(node);
                        view.dispatch(transaction)
                    };
                    reader.readAsDataURL(image)
                }

            }
        }
        return false;
    },
    handleDOMEvents: {
@slava-vishnyakov
Copy link
Author

Thanks, @bbscout, fixed it!

@ramsane
Copy link

ramsane commented Mar 10, 2021

@bbscout I am not clear with moving upload function to vue component. which vue component are we talking here.

Could you please elaborate little bit more ??

In slow networks, when we upload some image, it just stays blank which is not user friendly.

@bbscout
Copy link

bbscout commented Mar 10, 2021

@bbscout I am not clear with moving upload function to vue component. which vue component are we talking here.

Could you please elaborate little bit more ??

In slow networks, when we upload some image, it just stays blank which is not user friendly.

I'm not sure what is your question. You should better ask @slava-vishnyakov as an author of the gist. I just used his step by step manual to create modal to upload image.

@ramsane
Copy link

ramsane commented Mar 10, 2021

I am trying to put a image loader in src, till it get's posted to the backend and update that with the actual image url. In the previous comment, it seems to have solved, But can't get the exact way.

Hope this is clear now.

@MarcelloTheArcane
Copy link

When I paste or drop the image to the editor, the code correctly uploads it to the server. However, I get two copies of the image, the original one and the uploaded one. Any ideas?

I think you're right. When we upload image we get back source, both in case of drop and paste. But in case of paste we already have one image added by text editor automatically and we are also getting back one from server thus 2 images.
This only happens when copying image from website

@singhgursharnbir, @neelay92

Actually it's because the paste event from a website is two items - the actual image, and a html image tag. Here, I'm logging items from const items = (event.clipboardData || event.originalEvent.clipboardData).items, and it shows two when pasted from a website and one when pasted from the filesystem.

image

And the item's type:
image

You end up with two images embedded - one uploaded to your server, and the other as a copy directly from the website's image source.

I think this HTML paste is being handled elsewhere by Tiptap, and I'm not sure how you can turn it off without turning off paste for text and everything else too ☹️

@MarcelloTheArcane
Copy link

Here's a relevant issue from another repository: kensnyder/quill-image-drop-module#7

@captenmasin
Copy link

This is dope and exactly what I needed so thank you <3

Don't suppose anyone has written out a solution to be able to select the image by clicking the icon button?

@tabdon
Copy link

tabdon commented May 6, 2021

Has anyone ported this over to TipTap 2.0 code?

@jesster2k10
Copy link

Has anyone ported this over to TipTap 2.0 code?

Yeah! I've gotten it to work with TipTap 2.0 using the following (and typescript):

import { Node, nodeInputRule } from '@tiptap/core';
import { dropImagePlugin, UploadFn } from './drop_image';

/**
 * Matches following attributes in Markdown-typed image: [, alt, src, title]
 *
 * Example:
 * ![Lorem](image.jpg) -> [, "Lorem", "image.jpg"]
 * ![](image.jpg "Ipsum") -> [, "", "image.jpg", "Ipsum"]
 * ![Lorem](image.jpg "Ipsum") -> [, "Lorem", "image.jpg", "Ipsum"]
 */
const IMAGE_INPUT_REGEX = /!\[(.+|:?)\]\((\S+)(?:(?:\s+)["'](\S+)["'])?\)/;

export const createImageExtension = (uploadFn: UploadFn) => {
  return Node.create({
    name: 'image',
    inline: true,
    group: 'inline',
    draggable: true,
    addAttributes: () => ({
      src: {},
      alt: { default: null },
      title: { default: null },
    }),
    parseHTML: () => [
      {
        tag: 'img[src]',
        getAttrs: (dom) => {
          if (typeof dom === 'string') return {};
          const element = dom as HTMLImageElement;

          return {
            src: element.getAttribute('src'),
            title: element.getAttribute('title'),
            alt: element.getAttribute('alt'),
          };
        },
      },
    ],
    renderHTML: ({ HTMLAttributes }) => ['img', HTMLAttributes],

    // @ts-ignore
    addCommands() {
      return (attrs) => (state, dispatch) => {
        const { selection } = state;
        const position = selection.$cursor
          ? selection.$cursor.pos
          : selection.$to.pos;
        const node = this.type.create(attrs);
        const transaction = state.tr.insert(position, node);
        dispatch(transaction);
      };
    },
    addInputRules() {
      return [
        nodeInputRule(IMAGE_INPUT_REGEX, this.type, (match) => {
          const [, alt, src, title] = match;
          return {
            src,
            alt,
            title,
          };
        }),
      ];
    },
    addProseMirrorPlugins() {
      return [dropImagePlugin(uploadFn)];
    },
  });
};
import { Plugin, PluginKey } from 'prosemirror-state';

export type UploadFn = (image: File) => Promise<string>;

export const dropImagePlugin = (upload: UploadFn) => {
  return new Plugin({
    props: {
      handlePaste(view, event, slice) {
        const items = Array.from(event.clipboardData?.items || []);
        const { schema } = view.state;

        items.forEach((item) => {
          const image = item.getAsFile();

          if (item.type.indexOf('image') === 0) {
            event.preventDefault();

            if (upload && image) {
              upload(image).then((src) => {
                const node = schema.nodes.image.create({
                  src: src,
                });
                const transaction = view.state.tr.replaceSelectionWith(node);
                view.dispatch(transaction);
              });
            }
          } else {
            const reader = new FileReader();
            reader.onload = (readerEvent) => {
              const node = schema.nodes.image.create({
                src: readerEvent.target?.result,
              });
              const transaction = view.state.tr.replaceSelectionWith(node);
              view.dispatch(transaction);
            };
            if (!image) return;
            reader.readAsDataURL(image);
          }
        });

        return false;
      },
      handleDOMEvents: {
        drop: (view, event) => {
          const hasFiles =
            event.dataTransfer &&
            event.dataTransfer.files &&
            event.dataTransfer.files.length;

          if (!hasFiles) {
            return false;
          }

          const images = Array.from(
            event.dataTransfer?.files ?? []
          ).filter((file) => /image/i.test(file.type));

          if (images.length === 0) {
            return false;
          }

          event.preventDefault();

          const { schema } = view.state;
          const coordinates = view.posAtCoords({
            left: event.clientX,
            top: event.clientY,
          });
          if (!coordinates) return false;

          images.forEach(async (image) => {
            const reader = new FileReader();

            if (upload) {
              const node = schema.nodes.image.create({
                src: await upload(image),
              });
              const transaction = view.state.tr.insert(coordinates.pos, node);
              view.dispatch(transaction);
            } else {
              reader.onload = (readerEvent) => {
                const node = schema.nodes.image.create({
                  src: readerEvent.target?.result,
                });
                const transaction = view.state.tr.insert(coordinates.pos, node);
                view.dispatch(transaction);
              };
              reader.readAsDataURL(image);
            }
          });

          return true;
        },
      },
    },
  });
};

@tabdon
Copy link

tabdon commented May 24, 2021

@jesster2k10 thank you so much!

@captenmasin
Copy link

When I paste or drop the image to the editor, the code correctly uploads it to the server. However, I get two copies of the image, the original one and the uploaded one. Any ideas?

I think you're right. When we upload image we get back source, both in case of drop and paste. But in case of paste we already have one image added by text editor automatically and we are also getting back one from server thus 2 images.
This only happens when copying image from website

@singhgursharnbir, @neelay92

Actually it's because the paste event from a website is two items - the actual image, and a html image tag. Here, I'm logging items from const items = (event.clipboardData || event.originalEvent.clipboardData).items, and it shows two when pasted from a website and one when pasted from the filesystem.

image

And the item's type:
image

You end up with two images embedded - one uploaded to your server, and the other as a copy directly from the website's image source.

I think this HTML paste is being handled elsewhere by Tiptap, and I'm not sure how you can turn it off without turning off paste for text and everything else too ☹️

Has anyone managed to figure this out? I've Googled around but unfortunately I'm relatively new to the JS-world

@jslim89
Copy link

jslim89 commented Jun 4, 2021

This extension works great with paste event & drop event.
I'm adding an additional button to let user click & upload

<v-btn @click="fileSelected" small plain>
    <v-icon>mdi-image-plus</v-icon>
    <input type="file"
        class="image-input"
        @change="fileSelected">
</v-btn>

...
export default {
    methods: {
        fileSelected: function (evt) {
            // how can I trigger this image extension?
        },
    },
}

Anyone deal with this before?

@waptik
Copy link

waptik commented Jul 11, 2021

Has anyone ported this over to TipTap 2.0 code?

Yeah! I've gotten it to work with TipTap 2.0 using the following (and typescript):

import { Node, nodeInputRule } from '@tiptap/core';
import { dropImagePlugin, UploadFn } from './drop_image';

/**
 * Matches following attributes in Markdown-typed image: [, alt, src, title]
 *
 * Example:
 * ![Lorem](image.jpg) -> [, "Lorem", "image.jpg"]
 * ![](image.jpg "Ipsum") -> [, "", "image.jpg", "Ipsum"]
 * ![Lorem](image.jpg "Ipsum") -> [, "Lorem", "image.jpg", "Ipsum"]
 */
const IMAGE_INPUT_REGEX = /!\[(.+|:?)\]\((\S+)(?:(?:\s+)["'](\S+)["'])?\)/;

export const createImageExtension = (uploadFn: UploadFn) => {
  return Node.create({
    name: 'image',
    inline: true,
    group: 'inline',
    draggable: true,
    addAttributes: () => ({
      src: {},
      alt: { default: null },
      title: { default: null },
    }),
    parseHTML: () => [
      {
        tag: 'img[src]',
        getAttrs: (dom) => {
          if (typeof dom === 'string') return {};
          const element = dom as HTMLImageElement;

          return {
            src: element.getAttribute('src'),
            title: element.getAttribute('title'),
            alt: element.getAttribute('alt'),
          };
        },
      },
    ],
    renderHTML: ({ HTMLAttributes }) => ['img', HTMLAttributes],

    // @ts-ignore
    addCommands() {
      return (attrs) => (state, dispatch) => {
        const { selection } = state;
        const position = selection.$cursor
          ? selection.$cursor.pos
          : selection.$to.pos;
        const node = this.type.create(attrs);
        const transaction = state.tr.insert(position, node);
        dispatch(transaction);
      };
    },
    addInputRules() {
      return [
        nodeInputRule(IMAGE_INPUT_REGEX, this.type, (match) => {
          const [, alt, src, title] = match;
          return {
            src,
            alt,
            title,
          };
        }),
      ];
    },
    addProseMirrorPlugins() {
      return [dropImagePlugin(uploadFn)];
    },
  });
};
import { Plugin, PluginKey } from 'prosemirror-state';

export type UploadFn = (image: File) => Promise<string>;

export const dropImagePlugin = (upload: UploadFn) => {
  return new Plugin({
    props: {
      handlePaste(view, event, slice) {
        const items = Array.from(event.clipboardData?.items || []);
        const { schema } = view.state;

        items.forEach((item) => {
          const image = item.getAsFile();

          if (item.type.indexOf('image') === 0) {
            event.preventDefault();

            if (upload && image) {
              upload(image).then((src) => {
                const node = schema.nodes.image.create({
                  src: src,
                });
                const transaction = view.state.tr.replaceSelectionWith(node);
                view.dispatch(transaction);
              });
            }
          } else {
            const reader = new FileReader();
            reader.onload = (readerEvent) => {
              const node = schema.nodes.image.create({
                src: readerEvent.target?.result,
              });
              const transaction = view.state.tr.replaceSelectionWith(node);
              view.dispatch(transaction);
            };
            if (!image) return;
            reader.readAsDataURL(image);
          }
        });

        return false;
      },
      handleDOMEvents: {
        drop: (view, event) => {
          const hasFiles =
            event.dataTransfer &&
            event.dataTransfer.files &&
            event.dataTransfer.files.length;

          if (!hasFiles) {
            return false;
          }

          const images = Array.from(
            event.dataTransfer?.files ?? []
          ).filter((file) => /image/i.test(file.type));

          if (images.length === 0) {
            return false;
          }

          event.preventDefault();

          const { schema } = view.state;
          const coordinates = view.posAtCoords({
            left: event.clientX,
            top: event.clientY,
          });
          if (!coordinates) return false;

          images.forEach(async (image) => {
            const reader = new FileReader();

            if (upload) {
              const node = schema.nodes.image.create({
                src: await upload(image),
              });
              const transaction = view.state.tr.insert(coordinates.pos, node);
              view.dispatch(transaction);
            } else {
              reader.onload = (readerEvent) => {
                const node = schema.nodes.image.create({
                  src: readerEvent.target?.result,
                });
                const transaction = view.state.tr.insert(coordinates.pos, node);
                view.dispatch(transaction);
              };
              reader.readAsDataURL(image);
            }
          });

          return true;
        },
      },
    },
  });
};

Hi. I was looking for a way to use tiptap v2 with image upload (ueberdosis/tiptap#819) in react and came across this comment.
Can you please guide me as to how to use it in react or precisely how which command to call in order to invoke this extension?
Also it seems there's some typedef error with addCommands()

UPDATE

Finally got it working: https://gist.github.com/waptik/f44b0d3c803fade75456817b1b1df6b4

@HZ-labs
Copy link

HZ-labs commented Sep 14, 2021

When I paste or drop the image to the editor, the code correctly uploads it to the server. However, I get two copies of the image, the original one and the uploaded one. Any ideas?

I think you're right. When we upload image we get back source, both in case of drop and paste. But in case of paste we already have one image added by text editor automatically and we are also getting back one from server thus 2 images.
This only happens when copying image from website

@singhgursharnbir, @neelay92
Actually it's because the paste event from a website is two items - the actual image, and a html image tag. Here, I'm logging items from const items = (event.clipboardData || event.originalEvent.clipboardData).items, and it shows two when pasted from a website and one when pasted from the filesystem.
image
And the item's type:
image
You end up with two images embedded - one uploaded to your server, and the other as a copy directly from the website's image source.
I think this HTML paste is being handled elsewhere by Tiptap, and I'm not sure how you can turn it off without turning off paste for text and everything else too ☹️

Has anyone managed to figure this out? I've Googled around but unfortunately I'm relatively new to the JS-world

I found solution:
don't use - handlePaste
you need that -

 handleDOMEvents: {
    paste(view, event) {
        ...
        items.forEach((item) => {
            const file = item.getAsFile();
            if (file) {
                event.preventDefault();
                // upload file to server and insert to editor
            }
        }
        return false;
    }
}

@danline
Copy link

danline commented Oct 15, 2021

This code saved me a lot of time so thank you! I kept getting an error in tiptap 2.0 "find is not a function". I changed the addInputRules function to include the right format the tiptap core now expects and it worked.

    addInputRules() {
        return [
            nodeInputRule({
                find: inputRegex,
                type: this.type,
                getAttributes: match => {
                  const [,, alt, src, title] = match
        
                  return { src, alt, title }
                },
            }),
        ];
    },

@healer-1205
Copy link

Hello, slava
Thanks for your posting, It is very useful.
But when I run your code, there happens an error.
Like this:
image

So why this error happens? I want your help.
Thanks

@GunGriM2
Copy link

GunGriM2 commented Jan 3, 2022

@healer-1205 i believe this error happens when you incorrectly add extension to an Editor.
Maybe you forgot to set an upload function to the extension.

new Editor({
  extensions: [
    ...
    Image**(upload)**,
    ...

@theonmt
Copy link

theonmt commented Mar 1, 2022

My solution for Tiptap 2.0:
I created this CustomEventHandlers extension

import { Extension } from '@tiptap/core';
import { Plugin, PluginKey } from 'prosemirror-state';

// convert a blob to base64
const blobToBase64 = async blob => {
  return await new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.readAsDataURL(blob);
    reader.onload = () => resolve(reader.result);
    reader.onerror = error => reject(error);
  });
};

const CustomEventHandlers = Extension.create({
  name: 'CustomEventHandlers',
  
  addProseMirrorPlugins() {
  return [
    new Plugin({
    key: new PluginKey('customEventHandlers'),
    props: {
      handleDOMEvents: {
        drop: async (view, event) => {
          let files = event.dataTransfer.files;
          
          if (files.length) {
            event.preventDefault();
            const src = await blobToBase64(files[0]);
            src && this.editor.chain().focus().setImage({ src }).run();
          }
        },
      },
    },
    }),
  ];
  },
});

export default CustomEventHandlers;
export { CustomEventHandlers };

@LondonAppDev
Copy link

LondonAppDev commented Jun 17, 2022

Thanks a lot for the guide, this is really useful. One question, when using typescript, it shows a warning on the following:

Property 'dataTransfer' does not exist on type 'Event'

Relating to the dataTransfer properties being access on the event here:

handleDOMEvents: {
    drop(view, event) {
        const hasFiles = event.dataTransfer
            && event.dataTransfer.files
            && event.dataTransfer.files.length;

Anyone know how to do this properly in TypeScript?

Thanks in advance.

@WANZARGEN
Copy link

WANZARGEN commented Aug 3, 2022

@LondonAppDev I solved warning as below:

handleDOMEvents: {
            drop: (view, _event: Event) => {
                // event param must be Event type. don't change it to DragEvent type.
                const event = _event as DragEvent;
                const hasFiles = !!event.dataTransfer?.files?.length;

Let me know if anyone has a better way 🙏

@varun-raj
Copy link

I found solution: don't use - handlePaste you need that -

This saved my day! Thanks @HZ-labs

@claide
Copy link

claide commented Oct 16, 2023

Has anyone ported this over to TipTap 2.0 code?

Yeah! I've gotten it to work with TipTap 2.0 using the following (and typescript):

import { Node, nodeInputRule } from '@tiptap/core';
import { dropImagePlugin, UploadFn } from './drop_image';

/**
 * Matches following attributes in Markdown-typed image: [, alt, src, title]
 *
 * Example:
 * ![Lorem](image.jpg) -> [, "Lorem", "image.jpg"]
 * ![](image.jpg "Ipsum") -> [, "", "image.jpg", "Ipsum"]
 * ![Lorem](image.jpg "Ipsum") -> [, "Lorem", "image.jpg", "Ipsum"]
 */
const IMAGE_INPUT_REGEX = /!\[(.+|:?)\]\((\S+)(?:(?:\s+)["'](\S+)["'])?\)/;

export const createImageExtension = (uploadFn: UploadFn) => {
  return Node.create({
    name: 'image',
    inline: true,
    group: 'inline',
    draggable: true,
    addAttributes: () => ({
      src: {},
      alt: { default: null },
      title: { default: null },
    }),
    parseHTML: () => [
      {
        tag: 'img[src]',
        getAttrs: (dom) => {
          if (typeof dom === 'string') return {};
          const element = dom as HTMLImageElement;

          return {
            src: element.getAttribute('src'),
            title: element.getAttribute('title'),
            alt: element.getAttribute('alt'),
          };
        },
      },
    ],
    renderHTML: ({ HTMLAttributes }) => ['img', HTMLAttributes],

    // @ts-ignore
    addCommands() {
      return (attrs) => (state, dispatch) => {
        const { selection } = state;
        const position = selection.$cursor
          ? selection.$cursor.pos
          : selection.$to.pos;
        const node = this.type.create(attrs);
        const transaction = state.tr.insert(position, node);
        dispatch(transaction);
      };
    },
    addInputRules() {
      return [
        nodeInputRule(IMAGE_INPUT_REGEX, this.type, (match) => {
          const [, alt, src, title] = match;
          return {
            src,
            alt,
            title,
          };
        }),
      ];
    },
    addProseMirrorPlugins() {
      return [dropImagePlugin(uploadFn)];
    },
  });
};
import { Plugin, PluginKey } from 'prosemirror-state';

export type UploadFn = (image: File) => Promise<string>;

export const dropImagePlugin = (upload: UploadFn) => {
  return new Plugin({
    props: {
      handlePaste(view, event, slice) {
        const items = Array.from(event.clipboardData?.items || []);
        const { schema } = view.state;

        items.forEach((item) => {
          const image = item.getAsFile();

          if (item.type.indexOf('image') === 0) {
            event.preventDefault();

            if (upload && image) {
              upload(image).then((src) => {
                const node = schema.nodes.image.create({
                  src: src,
                });
                const transaction = view.state.tr.replaceSelectionWith(node);
                view.dispatch(transaction);
              });
            }
          } else {
            const reader = new FileReader();
            reader.onload = (readerEvent) => {
              const node = schema.nodes.image.create({
                src: readerEvent.target?.result,
              });
              const transaction = view.state.tr.replaceSelectionWith(node);
              view.dispatch(transaction);
            };
            if (!image) return;
            reader.readAsDataURL(image);
          }
        });

        return false;
      },
      handleDOMEvents: {
        drop: (view, event) => {
          const hasFiles =
            event.dataTransfer &&
            event.dataTransfer.files &&
            event.dataTransfer.files.length;

          if (!hasFiles) {
            return false;
          }

          const images = Array.from(
            event.dataTransfer?.files ?? []
          ).filter((file) => /image/i.test(file.type));

          if (images.length === 0) {
            return false;
          }

          event.preventDefault();

          const { schema } = view.state;
          const coordinates = view.posAtCoords({
            left: event.clientX,
            top: event.clientY,
          });
          if (!coordinates) return false;

          images.forEach(async (image) => {
            const reader = new FileReader();

            if (upload) {
              const node = schema.nodes.image.create({
                src: await upload(image),
              });
              const transaction = view.state.tr.insert(coordinates.pos, node);
              view.dispatch(transaction);
            } else {
              reader.onload = (readerEvent) => {
                const node = schema.nodes.image.create({
                  src: readerEvent.target?.result,
                });
                const transaction = view.state.tr.insert(coordinates.pos, node);
                view.dispatch(transaction);
              };
              reader.readAsDataURL(image);
            }
          });

          return true;
        },
      },
    },
  });
};

How do you use it on the editor? is it needs to be added to the editor extensions?

@dan-cooke
Copy link

Heres a full extension that has the following features

  1. Uses data url immediately to render the image while uploading for faster UX
  2. Uploads the image to your API in the background via custom react component
  3. Swaps the data-url image for your uploaded URL once its ready

I will be refining this and adding a tonne of features to it like resizing and cropping, will release as open source when its ready

import { useFilesControllerUpload } from '@templi/sdk';
import { mergeAttributes, Node } from '@tiptap/core';
import { Plugin, PluginKey } from '@tiptap/pm/state';
import { NodeViewWrapper, ReactNodeViewRenderer } from '@tiptap/react';
import { useEffect, useState } from 'react';

export interface ImageOptions {
  HTMLAttributes: Record<string, any>;
}

const blobToBase64 = async (blob: Blob): Promise<string> => {
  return await new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.readAsDataURL(blob);
    reader.onload = () => resolve(reader.result);
    reader.onerror = (error) => reject(error);
  });
};
declare module '@tiptap/core' {
  interface Commands<ReturnType> {
    image: {
      /**
       * Add an image
       */
      setImage: (options: {
        src?: string;
        alt?: string;
        title?: string;
      }) => ReturnType;
    };
  }
}

type ImageComponentProps = {
  node: {
    attrs: {
      src: string;
    };
  };
};
function ImageComponent(props: ImageComponentProps) {
  const { node } = props;

  // swap this for your API file upload code
  const { mutateAsync: upload, submittedAt } = useFilesControllerUpload();

  const [src, setSrc] = useState<string | undefined>(node?.attrs?.src);

  useEffect(() => {
    if (node.attrs.src.startsWith('data:') && !submittedAt) {
      async function uploadImage() {
        const formData = new FormData();
        const base64 = node.attrs.src.split(',')[1];
        const file = window.atob(base64);

        formData.set(
          'file',
          new Blob([file], { type: 'image/png' }),
          'image.png'
        );

        const uploadedFile = await upload({
          body: formData,
          headers: {
            'Content-Type': 'multipart/form-data',
          },
        });

        if (!uploadedFile) return;
        setSrc(uploadedFile.url);
      }

      uploadImage();
    }
  }, [node.attrs.src]);

  return (
    <NodeViewWrapper className="w-full">
      <img src={src} className="w-full" />
    </NodeViewWrapper>
  );
}

export const Image = Node.create<ImageOptions>({
  name: 'image',

  addOptions() {
    return {
      inline: false,
      allowBase64: false,
      HTMLAttributes: {},
    };
  },

  inline: false,
  group: 'block',

  draggable: true,

  addAttributes() {
    return {
      src: {
        default: null,
      },
      alt: {
        default: null,
      },
      title: {
        default: null,
      },
    };
  },

  parseHTML() {
    return [
      {
        tag: 'img[src]:not([src^="data:"])',
      },
    ];
  },

  renderHTML({ HTMLAttributes }) {
    return [
      'img',
      mergeAttributes(this.options.HTMLAttributes, HTMLAttributes),
    ];
  },

  addNodeView() {
    return ReactNodeViewRenderer(ImageComponent);
  },

  addCommands() {
    return {
      setImage:
        (options) =>
          ({ commands }) => {
            return commands.insertContent({
              type: this.name,
              attrs: options,
            });
          },
    };
  },

  addProseMirrorPlugins() {
    return [
      new Plugin({
        key: new PluginKey('imageDrop'),
        props: {
          handleDOMEvents: {
            drop: async (view, event) => {
              if (event?.dataTransfer?.files) {
                const files = event.dataTransfer.files;
                const file = files.item(0);

                if (file && file.type.includes('image')) {
                  const dataUrl = await blobToBase64(file);
                  return this.editor.chain().setImage({ src: dataUrl }).run();
                }
              }
              return false;
            },
          },
        },
      }),
    ];
  },
});

@xiangshu233
Copy link

@dan-cooke That's great, but I would like to ask if there is a vue version, thanks

@r614
Copy link

r614 commented Apr 1, 2024

@dan-cooke thanks for the snippet, happy to help with the extension if needed

@craftogrammer
Copy link

Heres a full extension that has the following features

  1. Uses data url immediately to render the image while uploading for faster UX
  2. Uploads the image to your API in the background via custom react component
  3. Swaps the data-url image for your uploaded URL once its ready

I will be refining this and adding a tonne of features to it like resizing and cropping, will release as open source when its ready

import { useFilesControllerUpload } from '@templi/sdk';
import { mergeAttributes, Node } from '@tiptap/core';
import { Plugin, PluginKey } from '@tiptap/pm/state';
import { NodeViewWrapper, ReactNodeViewRenderer } from '@tiptap/react';
import { useEffect, useState } from 'react';

export interface ImageOptions {
  HTMLAttributes: Record<string, any>;
}

const blobToBase64 = async (blob: Blob): Promise<string> => {
  return await new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.readAsDataURL(blob);
    reader.onload = () => resolve(reader.result);
    reader.onerror = (error) => reject(error);
  });
};
declare module '@tiptap/core' {
  interface Commands<ReturnType> {
    image: {
      /**
       * Add an image
       */
      setImage: (options: {
        src?: string;
        alt?: string;
        title?: string;
      }) => ReturnType;
    };
  }
}

type ImageComponentProps = {
  node: {
    attrs: {
      src: string;
    };
  };
};
function ImageComponent(props: ImageComponentProps) {
  const { node } = props;

  // swap this for your API file upload code
  const { mutateAsync: upload, submittedAt } = useFilesControllerUpload();

  const [src, setSrc] = useState<string | undefined>(node?.attrs?.src);

  useEffect(() => {
    if (node.attrs.src.startsWith('data:') && !submittedAt) {
      async function uploadImage() {
        const formData = new FormData();
        const base64 = node.attrs.src.split(',')[1];
        const file = window.atob(base64);

        formData.set(
          'file',
          new Blob([file], { type: 'image/png' }),
          'image.png'
        );

        const uploadedFile = await upload({
          body: formData,
          headers: {
            'Content-Type': 'multipart/form-data',
          },
        });

        if (!uploadedFile) return;
        setSrc(uploadedFile.url);
      }

      uploadImage();
    }
  }, [node.attrs.src]);

  return (
    <NodeViewWrapper className="w-full">
      <img src={src} className="w-full" />
    </NodeViewWrapper>
  );
}

export const Image = Node.create<ImageOptions>({
  name: 'image',

  addOptions() {
    return {
      inline: false,
      allowBase64: false,
      HTMLAttributes: {},
    };
  },

  inline: false,
  group: 'block',

  draggable: true,

  addAttributes() {
    return {
      src: {
        default: null,
      },
      alt: {
        default: null,
      },
      title: {
        default: null,
      },
    };
  },

  parseHTML() {
    return [
      {
        tag: 'img[src]:not([src^="data:"])',
      },
    ];
  },

  renderHTML({ HTMLAttributes }) {
    return [
      'img',
      mergeAttributes(this.options.HTMLAttributes, HTMLAttributes),
    ];
  },

  addNodeView() {
    return ReactNodeViewRenderer(ImageComponent);
  },

  addCommands() {
    return {
      setImage:
        (options) =>
          ({ commands }) => {
            return commands.insertContent({
              type: this.name,
              attrs: options,
            });
          },
    };
  },

  addProseMirrorPlugins() {
    return [
      new Plugin({
        key: new PluginKey('imageDrop'),
        props: {
          handleDOMEvents: {
            drop: async (view, event) => {
              if (event?.dataTransfer?.files) {
                const files = event.dataTransfer.files;
                const file = files.item(0);

                if (file && file.type.includes('image')) {
                  const dataUrl = await blobToBase64(file);
                  return this.editor.chain().setImage({ src: dataUrl }).run();
                }
              }
              return false;
            },
          },
        },
      }),
    ];
  },
});

thank you so much man!

@lokeshfitsys
Copy link

Heres a full extension that has the following features

  1. Uses data url immediately to render the image while uploading for faster UX
  2. Uploads the image to your API in the background via custom react component
  3. Swaps the data-url image for your uploaded URL once its ready

I will be refining this and adding a tonne of features to it like resizing and cropping, will release as open source when its ready

import { useFilesControllerUpload } from '@templi/sdk';
import { mergeAttributes, Node } from '@tiptap/core';
import { Plugin, PluginKey } from '@tiptap/pm/state';
import { NodeViewWrapper, ReactNodeViewRenderer } from '@tiptap/react';
import { useEffect, useState } from 'react';

export interface ImageOptions {
  HTMLAttributes: Record<string, any>;
}

const blobToBase64 = async (blob: Blob): Promise<string> => {
  return await new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.readAsDataURL(blob);
    reader.onload = () => resolve(reader.result);
    reader.onerror = (error) => reject(error);
  });
};
declare module '@tiptap/core' {
  interface Commands<ReturnType> {
    image: {
      /**
       * Add an image
       */
      setImage: (options: {
        src?: string;
        alt?: string;
        title?: string;
      }) => ReturnType;
    };
  }
}

type ImageComponentProps = {
  node: {
    attrs: {
      src: string;
    };
  };
};
function ImageComponent(props: ImageComponentProps) {
  const { node } = props;

  // swap this for your API file upload code
  const { mutateAsync: upload, submittedAt } = useFilesControllerUpload();

  const [src, setSrc] = useState<string | undefined>(node?.attrs?.src);

  useEffect(() => {
    if (node.attrs.src.startsWith('data:') && !submittedAt) {
      async function uploadImage() {
        const formData = new FormData();
        const base64 = node.attrs.src.split(',')[1];
        const file = window.atob(base64);

        formData.set(
          'file',
          new Blob([file], { type: 'image/png' }),
          'image.png'
        );

        const uploadedFile = await upload({
          body: formData,
          headers: {
            'Content-Type': 'multipart/form-data',
          },
        });

        if (!uploadedFile) return;
        setSrc(uploadedFile.url);
      }

      uploadImage();
    }
  }, [node.attrs.src]);

  return (
    <NodeViewWrapper className="w-full">
      <img src={src} className="w-full" />
    </NodeViewWrapper>
  );
}

export const Image = Node.create<ImageOptions>({
  name: 'image',

  addOptions() {
    return {
      inline: false,
      allowBase64: false,
      HTMLAttributes: {},
    };
  },

  inline: false,
  group: 'block',

  draggable: true,

  addAttributes() {
    return {
      src: {
        default: null,
      },
      alt: {
        default: null,
      },
      title: {
        default: null,
      },
    };
  },

  parseHTML() {
    return [
      {
        tag: 'img[src]:not([src^="data:"])',
      },
    ];
  },

  renderHTML({ HTMLAttributes }) {
    return [
      'img',
      mergeAttributes(this.options.HTMLAttributes, HTMLAttributes),
    ];
  },

  addNodeView() {
    return ReactNodeViewRenderer(ImageComponent);
  },

  addCommands() {
    return {
      setImage:
        (options) =>
          ({ commands }) => {
            return commands.insertContent({
              type: this.name,
              attrs: options,
            });
          },
    };
  },

  addProseMirrorPlugins() {
    return [
      new Plugin({
        key: new PluginKey('imageDrop'),
        props: {
          handleDOMEvents: {
            drop: async (view, event) => {
              if (event?.dataTransfer?.files) {
                const files = event.dataTransfer.files;
                const file = files.item(0);

                if (file && file.type.includes('image')) {
                  const dataUrl = await blobToBase64(file);
                  return this.editor.chain().setImage({ src: dataUrl }).run();
                }
              }
              return false;
            },
          },
        },
      }),
    ];
  },
});

What is @templi/sdk? Wheren I can get it? I tried installing using yarn add @templi/sdk but it's saying package not found.

@dan-cooke
Copy link

dan-cooke commented Sep 4, 2024

@lokeshfitsys that is a private package, you replace the file upload code with your own file upload code.

Hence the comment i left

// swap this for your API file upload code
const { mutateAsync: upload, submittedAt } = useFilesControllerUpload();

You can't expect to just copy and paste this and have file uploads working with your infrastrcuture 😆

@lokeshfitsys
Copy link

@lokeshfitsys that is a private package, you replace the file upload code with your own file upload code.

Hence the comment i left

// swap this for your API file upload code
const { mutateAsync: upload, submittedAt } = useFilesControllerUpload();

You can't expect to just copy and paste this and have file uploads working with your infrastrcuture 😆

Got it, I am newbie to react. Thank you so much for the clarification. 👍 😁

@mohanlokesh
Copy link

@dan-cooke

I have modified few things on the code. For my situation I can use base64 directly on the editor without moving to server and I have added button for select image and added resize image option.

I would appreciate your feedback!

/* eslint-disable @next/next/no-img-element */
import { useEffect, useRef, useState } from 'react'

import { mergeAttributes, Node } from '@tiptap/core'
import { Plugin, PluginKey } from '@tiptap/pm/state'
import { NodeViewWrapper, ReactNodeViewRenderer } from '@tiptap/react'
import { Resizable } from 're-resizable'

import CustomIconButton from '@/@core/components/mui/IconButton'

export interface ImageOptions {
  HTMLAttributes: Record<string, any>
}

export const blobToBase64 = async (blob: Blob): Promise<string> => {
  return await new Promise((resolve, reject) => {
    const reader = new FileReader()

    reader.readAsDataURL(blob)
    reader.onload = () => resolve(reader.result as string)
    reader.onerror = error => reject(error)
  })
}

declare module '@tiptap/core' {
  interface Commands<ReturnType> {
    image: {
      setImage: (options: { src: string; alt?: string; title?: string; width?: string | undefined }) => ReturnType
    }
  }
}

type ImageComponentProps = {
  node: {
    attrs: {
      src: string
      alt?: string
      title?: string
      width?: string
    }
  }
  updateAttributes: (attrs: any) => void
}

function ImageComponent(props: ImageComponentProps) {
  const { node, updateAttributes } = props
  const [src, setSrc] = useState<string | undefined>(node?.attrs?.src)
  const [width, setWidth] = useState(node.attrs.width || '100%')

  useEffect(() => {
    if (node.attrs.src.startsWith('data:')) {
      setSrc(node.attrs.src)
    } else {
      async function convertImageToBase64() {
        const response = await fetch(node.attrs.src)
        const blob = await response.blob()
        const reader = new FileReader()

        reader.readAsDataURL(blob)

        reader.onloadend = () => {
          setSrc(reader.result as string)
        }
      }

      if (node.attrs.src) {
        convertImageToBase64()
      }
    }
  }, [node.attrs.src])

  const handleResize = (_e: any, _direction: any, ref: any) => {
    const newWidth = ref.style.width

    setWidth(newWidth)
    updateAttributes({ width: newWidth })
  }

  return (
    <NodeViewWrapper className='w-full'>
      <Resizable size={{ width }} onResizeStop={handleResize} style={{ display: 'inline-block' }}>
        <img src={src} style={{ width }} alt={node.attrs.alt || 'image'} />
      </Resizable>
    </NodeViewWrapper>
  )
}

export const Image = Node.create<ImageOptions>({
  name: 'image',

  addOptions() {
    return {
      inline: false,
      allowBase64: false,
      HTMLAttributes: {}
    }
  },

  inline: false,
  group: 'block',
  draggable: true,

  addAttributes() {
    return {
      src: { default: null },
      alt: { default: null },
      title: { default: null },
      width: { default: '100%' }
    }
  },

  parseHTML() {
    return [{ tag: 'img[src]:not([src^="data:"])' }]
  },

  renderHTML({ HTMLAttributes }) {
    return ['img', mergeAttributes(this.options.HTMLAttributes, HTMLAttributes)]
  },

  addNodeView() {
    return ReactNodeViewRenderer(ImageComponent)
  },

  addCommands() {
    return {
      setImage:
        options =>
        ({ commands }) => {
          return commands.insertContent({
            type: this.name,
            attrs: options
          })
        }
    }
  },

  addProseMirrorPlugins() {
    return [
      new Plugin({
        key: new PluginKey('imageDrop'),
        props: {
          handleDOMEvents: {
            drop: (_view, event) => {
              if (event?.dataTransfer?.files) {
                const files = event.dataTransfer.files
                const file = files.item(0)

                if (file && file.type.includes('image')) {
                  const handleDrop = async () => {
                    const dataUrl = await blobToBase64(file)

                    this.editor.chain().setImage({ src: dataUrl }).run()
                  }

                  handleDrop()

                  return true
                }
              }

              return false
            }
          }
        }
      })
    ]
  }
})

// Usage example for manual upload
export const ManualImageUpload = ({ editor }: { editor: any }) => {
  const fileInputRef = useRef<HTMLInputElement>(null)

  const handleImageUpload = async (event: any) => {
    const file = event.target.files[0]

    if (file && file.type.includes('image')) {
      const base64 = await blobToBase64(file)

      editor.chain().setImage({ src: base64 }).run()
    }
  }

  return (
    <>
      <input type='file' accept='image/*' ref={fileInputRef} style={{ display: 'none' }} onChange={handleImageUpload} />
      <CustomIconButton
        variant='text'
        title='Upload Image'
        size='small'
        color='secondary'
        onClick={() => fileInputRef.current?.click()}
      >
        <i className='ri-image-add-line'></i>
      </CustomIconButton>
    </>
  )
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment