The password prompt was made for system security so if you do this might make it vulnerable.
Create the Group group on your machine. or you can run this "sudo groupadd -r [Group]" You can any user you want to this system group by runing "sudo usermod -a -G [Group] [User]" Now you need to create our PolicyKit policy that will allow the users of Group to run virt-manager you will create a file at this path:"/etc/polkit-1/localauthority/50-local.d/50-org.Group-libvirt-local-access.pkla" and you will put lines below in it
[Allow group [Group] libvirt management permissions]