Skip to content

Instantly share code, notes, and snippets.

@ahwillia
Created September 11, 2016 06:19
Show Gist options
  • Save ahwillia/8d04471daf0a9a12794107dec8fe3884 to your computer and use it in GitHub Desktop.
Save ahwillia/8d04471daf0a9a12794107dec8fe3884 to your computer and use it in GitHub Desktop.
Prototype for alternating gradient descent with Optim.jl
using Optim
# low-dimensional data embedded in high-dimension
data = randn(100,5)*randn(5,100);
# a container for the parameters we fit
immutable PCA{T<:Real}
X::Matrix{T}
Y::Matrix{T}
end
PCA(m::Int,n::Int,r::Int) = PCA(randn(m,r),randn(r,n))
# initialize model
const p = PCA(100,100,5)
# total objective function is:
# f(X,Y) = f1(X) + f2(Y)
f1(X) = 0.5*sumabs2(data - X*p.Y)
f2(Y) = 0.5*sumabs2(data - p.X*Y)
# We will alternate between updating X and Y since
# the subproblems are convex and this may lead to
# better/quicker convergence
d1 = DifferentiableFunction(f1)
d2 = DifferentiableFunction(f2)
gd = GradientDescent()
o = OptimizationOptions()
s1 = Optim.initial_state(gd, o, d1, p.X)
s2 = Optim.initial_state(gd, o, d2, p.Y)
for iter = 1:iterations
Optim.update!(d1, s1, gd)
Optim.update!(d2, s2, gd)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment