NewDB.md

New www user database implementation

The goal of this new database implementation is to make the database more robust by adding a cockrochdb implementation and modifying the database interface and the current leveldb implementation as appropriate.

Design

Modify User structure

In order to support cockroachdb the User sensitive data must be encrypted at flight and at rest. It will require a few modifications in the Db design as part of the user information should be public and the rest encrypted.

In order to accomplish that the User struct will be splitted in two other structs UserDetails and User.

UserDetails: contains all the sensitive user data and and it is the part which will be enconded and encrypted before saving into the db

type UserDetails struct {
    Email                           string    // Email address + lookup key.
	HashedPassword                  []byte    // Blowfish hash
	Admin                           bool      // Is user an admin
	PaywallAddressIndex             uint64    // Sequential id used to generate paywall address
	NewUserPaywallAddress           string    // Address the user needs to send to
	NewUserPaywallAmount            uint64    // Amount the user needs to send
	NewUserPaywallTx                string    // Paywall transaction id
	NewUserPaywallTxNotBefore       int64     // Transactions occurring before this time will not be valid.
	NewUserPaywallPollExpiry        int64     // After this time, the user's paywall address will not be continuously polled
	NewUserVerificationToken        []byte    // New user registration verification token
	NewUserVerificationExpiry       int64     // New user registration verification expiration
	ResendNewUserVerificationExpiry int64     // Resend request for new user registration verification expiration
	UpdateKeyVerificationToken      []byte    // Verification token for updating keypair
	UpdateKeyVerificationExpiry     int64     // Verification expiration
	ResetPasswordVerificationToken  []byte    // Reset password token
	ResetPasswordVerificationExpiry int64     // Reset password token expiration
	LastLoginTime                   int64     // Unix timestamp of when the user last logged in
	FailedLoginAttempts             uint64    // Number of failed login a user has made in a row
	Deactivated                     bool      // Whether the account is deactivated or not
	EmailNotifications              uint64    // Notify the user via emails
	ProposalCommentsAccessTimes map[string]int64
	Identities []Identity
	ProposalPaywalls []ProposalPaywall
	UnspentProposalCredits []ProposalCredit
	SpentProposalCredits []ProposalCredit
}

User: contains the public information of the user as well the encrypted blob of UserDetails

type User struct {
    ID uuid.UUID // Primary key
    Username string // Secoondary key
    Payload []byte // Encrypted blob of user details
}

Update the interface

In order to support the changes in the User structure the Db interface will be modified accordingly.

type Database interface {
    UserGetByID(uuid.UUID) (string, UserDetails, error) // Returns the requested UserDetails along with the username
    UserGetByUsername(string) (uuid.UUID, UserDetails, error) // Returns the requested UserDetails along with the user ID
    UserNew(string, UserDetails) error // Creates a new user with the username and details provided
    UserUpdate(uuid.UUID, string, UserDetails) // Updates the user details and username if it has changed
    Open() error
    Close() error
}

Note that the work of encrypting/decrypting the user details is always hidden from the caller of the database interface methods.

package database

type User struct {
	ID       uuid.UUID // Primary key
	Username string    // Secondary key (index)
	Payload  []byte    // Encrypted blob of user details
}

type Database interface {
	UserGetByID(uuid.UUID) (*User, error)    
	UserGetByUsername(string) (*User, error) 
	UserNew(User) error                     
	UserUpdate(User) error                   
	Close() error
}

User package:

package user

import (
	"github.com/decred/politeia/politeiad/api/v1/identity"
	"github.com/decred/politeia/politeiawww/database"
	"github.com/google/uuid"
)

type userAPI struct {
	db     database.Database
	secret *EncryptionKey
}

// EncryptionKey wraps a key used for encrypting/decrypting the database
// data and the time when it was created.
type EncryptionKey struct {
	Key  [32]byte // Key used for encryption
	Time int64    // Time key was created
}

// Identity wraps an ed25519 public key and timestamps to indicate if it is
// active.  If deactivated != 0 then the key is no longer valid.
type Identity struct {
	Key         [identity.PublicKeySize]byte // ed25519 public key
	Activated   int64                        // Time key was activated for use
	Deactivated int64                        // Time key was deactivated
}

// ProposalPaywall allows the user to purchase proposal credits.  Proposal
// paywalls are only valid for one tx.  The number of proposal credits created
// is determined by dividing the tx amount by the credit price.  Proposal
// paywalls expire after a set duration. politeiawww polls the paywall address
// for a payment tx until the paywall is either paid or it expires.
type ProposalPaywall struct {
	ID          uint64 // Paywall ID
	CreditPrice uint64 // Cost per proposal credit in atoms
	Address     string // Paywall address
	TxNotBefore int64  // Minimum timestamp for paywall tx
	PollExpiry  int64  // After this time, the paywall address will not be continuously polled
	TxID        string // Payment transaction ID
	TxAmount    uint64 // Amount sent to paywall address in atoms
	NumCredits  uint64 // Number of proposal credits created by payment tx
}

// ProposalCredit allows the user to submit a new proposal.  Credits are
// created when a user sends a payment to a proposal paywall.  A credit is
// automatically spent when a user submits a new proposal.  When a credit is
// spent, it is updated with the proposal's censorship token and moved to the
// user's spent proposal credits list.
type ProposalCredit struct {
	PaywallID       uint64 // ID of the proposal paywall that created this credit
	Price           uint64 // Price this credit was purchased at in atoms
	DatePurchased   int64  // Unix timestamp of when the credit was purchased
	TxID            string // Payment transaction ID
	CensorshipToken string // Censorship token of proposal that used this credit
}

type User struct {
	ID                              uuid.UUID // Unique user uuid
	Email                           string    // Email address + lookup key.
	Username                        string    // Unique username
	HashedPassword                  []byte    // Blowfish hash
	Admin                           bool      // Is user an admin
	PaywallAddressIndex             uint64    // Sequential id used to generate paywall address
	NewUserPaywallAddress           string    // Address the user needs to send to
	NewUserPaywallAmount            uint64    // Amount the user needs to send
	NewUserPaywallTx                string    // Paywall transaction id
	NewUserPaywallTxNotBefore       int64     // Transactions occurring before this time will not be valid.
	NewUserPaywallPollExpiry        int64     // After this time, the user's paywall address will not be continuously polled
	NewUserVerificationToken        []byte    // New user registration verification token
	NewUserVerificationExpiry       int64     // New user registration verification expiration
	ResendNewUserVerificationExpiry int64     // Resend request for new user registration verification expiration
	UpdateKeyVerificationToken      []byte    // Verification token for updating keypair
	UpdateKeyVerificationExpiry     int64     // Verification expiration
	ResetPasswordVerificationToken  []byte    // Reset password token
	ResetPasswordVerificationExpiry int64     // Reset password token expiration
	LastLoginTime                   int64     // Unix timestamp of when the user last logged in
	FailedLoginAttempts             uint64    // Number of failed login a user has made in a row
	Deactivated                     bool      // Whether the account is deactivated or not
	EmailNotifications              uint64    // Notify the user via emails

	// Access times for proposal comments that have been accessed by the user.
	// Each string represents a proposal token, and the int64 represents the
	// time that the proposal has been most recently accessed in the format of
	// a UNIX timestamp.
	ProposalCommentsAccessTimes map[string]int64

	// All identities the user has ever used.  User should only have one
	// active key at a time.  We allow multiples in order to deal with key
	// loss.
	Identities []Identity

	// All proposal paywalls that have been issued to the user in chronological
	// order.
	ProposalPaywalls []ProposalPaywall

	// All proposal credits that have been purchased by the user, but have not
	// yet been used to submit a proposal.  Once a credit is used to submit a
	// proposal, it is updated with the proposal's censorship token and moved to
	// the user's spent proposal credits list.  The price that the proposal
	// credit was purchased at is in atoms.
	UnspentProposalCredits []ProposalCredit

	// All credits that have been purchased by the user and have already been
	// used to submit proposals.  Spent credits have a proposal censorship token
	// associated with them to signify that they have been spent. The price that
	// the proposal credit was purchased at is in atoms.
	SpentProposalCredits []ProposalCredit
}

func (ua *userAPI) UserNew(u User) error {
	// log.Tracef("UserNew: %v", u.Email)

	return nil
}

// UserGetByUsername returns a user record given its username, if found in the
// database.
func (ua *userAPI) UserGetByUsername(username string) (*User, error) {
	// log.Tracef("UserGetByUsername: %v", username)

	return nil, nil
}

// UserUpdate updates an existing user in the database
func (ua *userAPI) UserUpdate(u database.User) error {
	// log.Tracef("UserUpdate: %v", u.ID)

	return nil
}

// UserGet returns a user record if found in the database.
func (ua *userAPI) UserGetByEmail(email string) (*User, error) {
	// log.Tracef("UserGet: %v", email)

	return nil, nil
}

Database:

// Copyright (c) 2017 The Decred developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.

package database

import (
	"errors"

	"github.com/google/uuid"
)


// LastPaywallAddressIndex wraps the next paywall index to be used for
// the next user record inserted.
type LastPaywallAddressIndex struct {
	RecordType    RecordTypeT `json:"recordtype"`    // Record type
	RecordVersion uint32      `json:"recordversion"` // Database interface version

	Index uint64 `json:"index"`
}

// Version contains the database version.
type Version struct {
	RecordType    RecordTypeT `json:"recordtype"`    // Record type
	RecordVersion uint32      `json:"recordversion"` // Database interface version

	Version uint32 `json:"version"` // Database version
	Time    int64  `json:"time"`    // Time of record creation
}

type User struct {
	ID       uuid.UUID // Primary key
	Username string    // Secondary key (index)
	Payload  []byte    // Encrypted blob of user details
}

type Database interface {
	UserGetByID(uuid.UUID) (*User, error)
	UserGetByUsername(string) (*User, error)
	UserNew(User) error
	UserUpdate(User) error
	Close() error
}