Zelig January 2012

zelig@lists.gking.harvard.edu

8 participants
12 discussions

speed up 'sim' in Zelig when using ei.RxC

by Carolina Plescia

Dear all, I am using the model "ei.RxC" in Zelig, but unfortunately when I add covariates the “sim" function slows down terribly (more than 4 days for 20-30 simulations [n=20]). My intuition was to profile the sim function to find out which sub-function is the bottleneck using: Rprof("profilesim.out") out<--sim(...) summaryRprof("profilesim.out") Unfortunately I get the following error: summaryRprof("profilesim.out") Error in FUN(X[[556L]], ...) : subscript out of bounds Might it be that the resulting matrix is just too big to be printed by R? (The profilesim.out is nearly 70MB...) When I use the Rprof with the demo code in Zelig it looks like it is spending a lot of time in optim. data(Weimar) z.out2 <- zelig(cbind(Nazi, Government, Communists, FarRight, Other) ~ shareunemployed + shareblue + sharewhite + shareself + sharedomestic, covar = ~ shareprotestants, model = "ei.RxC", data = Weimar) x.out2 <- setx(z.out2) Rprof("simcall") s.out2 <- sim(z.out2, num = 10) Rprof(NULL) summaryRprof("simcall") $by.self self.time self.pct total.time total.pct "apply" 104.76 36.51 173.52 60.47 "fn" 54.54 19.01 285.48 99.48 "matrix" 10.42 3.63 25.00 8.71 "unlist" 10.10 3.52 25.30 8.82 "exp" 9.74 3.39 9.74 3.39 "as.vector" 9.28 3.23 14.40 5.02 "FUN" 8.94 3.12 9.06 3.16 "lapply" 8.66 3.02 15.34 5.35 "aperm" 6.84 2.38 6.84 2.38 "match.fun" 5.80 2.02 6.34 2.21 "sum" 5.24 1.83 5.24 1.83 "%*%" 4.34 1.51 4.34 1.51 "array" 3.80 1.32 4.90 1.71 "+" 3.72 1.30 3.72 1.30 "-" 3.70 1.29 3.70 1.29 "*" 3.54 1.23 3.54 1.23 "/" 2.76 0.96 2.76 0.96 "!" 2.70 0.94 2.70 0.94 "vector" 2.60 0.91 2.60 0.91 "prod" 2.52 0.88 2.52 0.88 "==" 2.44 0.85 2.44 0.85 "^" 2.34 0.82 2.34 0.82 "any" 2.22 0.77 2.22 0.77 "!=" 2.20 0.77 2.20 0.77 "is.vector" 2.18 0.76 2.18 0.76 "is.null" 1.64 0.57 1.64 0.57 "length" 1.52 0.53 1.52 0.53 "dim<-" 1.36 0.47 1.36 0.47 "<" 0.90 0.31 0.90 0.31 ":" 0.86 0.30 0.86 0.30 "is.object" 0.72 0.25 0.72 0.25 "is.function" 0.54 0.19 0.54 0.19 "<Anonymous>" 0.42 0.15 285.90 99.63 "strsplit" 0.40 0.14 0.40 0.14 "(" 0.34 0.12 0.34 0.12 "dim" 0.34 0.12 0.34 0.12 "is.character" 0.34 0.12 0.34 0.12 "is.recursive" 0.32 0.11 0.32 0.11 "dimnames" 0.30 0.10 0.30 0.10 "names<-" 0.28 0.10 0.28 0.10 "optim" 0.24 0.08 286.14 99.71 "names" 0.22 0.08 0.22 0.08 "seq_len" 0.22 0.08 0.22 0.08 "as.integer" 0.12 0.04 0.12 0.04 ">" 0.10 0.03 0.10 0.03 "readLines" 0.08 0.03 0.08 0.03 "unique.default" 0.06 0.02 0.10 0.03 "is.finite" 0.06 0.02 0.06 0.02 "inherits" 0.04 0.01 0.04 0.01 "is.numeric" 0.04 0.01 0.04 0.01 ".Call" 0.02 0.01 0.02 0.01 "is.list" 0.02 0.01 0.02 0.01 "match" 0.02 0.01 0.02 0.01 "model.frame.default" 0.02 0.01 0.02 0.01 "rownames" 0.02 0.01 0.02 0.01 "structure" 0.02 0.01 0.02 0.01 $by.total total.time total.pct self.time self.pct "sim" 286.28 99.76 0.00 0.00 "sim.eiRxC" 286.28 99.76 0.00 0.00 "boot" 286.24 99.75 0.00 0.00 "callparamseiestim" 286.24 99.75 0.00 0.00 "eval" 286.24 99.75 0.00 0.00 "statistic" 286.24 99.75 0.00 0.00 "zelig" 286.24 99.75 0.00 0.00 "zelig.default" 286.24 99.75 0.00 0.00 "paramsei.estim" 286.18 99.73 0.00 0.00 "optim" 286.14 99.71 0.24 0.08 "<Anonymous>" 285.90 99.63 0.42 0.15 "fn" 285.48 99.48 54.54 19.01 "apply" 173.52 60.47 104.76 36.51 "unlist" 25.30 8.82 10.10 3.52 "matrix" 25.00 8.71 10.42 3.63 "lapply" 15.34 5.35 8.66 3.02 "as.vector" 14.40 5.02 9.28 3.23 "exp" 9.74 3.39 9.74 3.39 "FUN" 9.06 3.16 8.94 3.12 "aperm" 6.84 2.38 6.84 2.38 "match.fun" 6.34 2.21 5.80 2.02 "sum" 5.24 1.83 5.24 1.83 "array" 4.90 1.71 3.80 1.32 "%*%" 4.34 1.51 4.34 1.51 "+" 3.72 1.30 3.72 1.30 "-" 3.70 1.29 3.70 1.29 "*" 3.54 1.23 3.54 1.23 "/" 2.76 0.96 2.76 0.96 "!" 2.70 0.94 2.70 0.94 "vector" 2.60 0.91 2.60 0.91 "prod" 2.52 0.88 2.52 0.88 "==" 2.44 0.85 2.44 0.85 "^" 2.34 0.82 2.34 0.82 "any" 2.22 0.77 2.22 0.77 "!=" 2.20 0.77 2.20 0.77 "is.vector" 2.18 0.76 2.18 0.76 "is.null" 1.64 0.57 1.64 0.57 "length" 1.52 0.53 1.52 0.53 "dim<-" 1.36 0.47 1.36 0.47 "<" 0.90 0.31 0.90 0.31 ":" 0.86 0.30 0.86 0.30 "is.object" 0.72 0.25 0.72 0.25 "summaryRprof" 0.66 0.23 0.00 0.00 "is.function" 0.54 0.19 0.54 0.19 "strsplit" 0.40 0.14 0.40 0.14 "(" 0.34 0.12 0.34 0.12 "dim" 0.34 0.12 0.34 0.12 "is.character" 0.34 0.12 0.34 0.12 "is.recursive" 0.32 0.11 0.32 0.11 "dimnames" 0.30 0.10 0.30 0.10 "names<-" 0.28 0.10 0.28 0.10 "names" 0.22 0.08 0.22 0.08 "seq_len" 0.22 0.08 0.22 0.08 "as.integer" 0.12 0.04 0.12 0.04 ">" 0.10 0.03 0.10 0.03 "unique.default" 0.10 0.03 0.06 0.02 "readLines" 0.08 0.03 0.08 0.03 "is.finite" 0.06 0.02 0.06 0.02 "cmei" 0.06 0.02 0.00 0.00 "inherits" 0.04 0.01 0.04 0.01 "is.numeric" 0.04 0.01 0.04 0.01 "as.matrix" 0.04 0.01 0.00 0.00 "calc.fractions" 0.04 0.01 0.00 0.00 "is.factor" 0.04 0.01 0.00 0.00 "qi" 0.04 0.01 0.00 0.00 "qi.eiRxC" 0.04 0.01 0.00 0.00 ".Call" 0.02 0.01 0.02 0.01 "is.list" 0.02 0.01 0.02 0.01 "match" 0.02 0.01 0.02 0.01 "model.frame.default" 0.02 0.01 0.02 0.01 "rownames" 0.02 0.01 0.02 0.01 "structure" 0.02 0.01 0.02 0.01 "[" 0.02 0.01 0.00 0.00 "as.matrix.data.frame" 0.02 0.01 0.00 0.00 "cbind" 0.02 0.01 0.00 0.00 "[.data.frame" 0.02 0.01 0.00 0.00 "factor" 0.02 0.01 0.00 0.00 "mean.default" 0.02 0.01 0.00 0.00 "model.frame" 0.02 0.01 0.00 0.00 "sapply" 0.02 0.01 0.00 0.00 "solve" 0.02 0.01 0.00 0.00 "solve.default" 0.02 0.01 0.00 0.00 "table" 0.02 0.01 0.00 0.00 "unique" 0.02 0.01 0.00 0.00 $sample.interval [1] 0.02 $sampling.time [1] 286.96 My questions are: 1) Why do I get the following error when I profile my code? summaryRprof("profilesim.out") Error in FUN(X[[556L]], ...) : subscript out of bounds 2) Can I assume from the demo example that the function that needs to be speeded up when I use my code is"optim"? 3) If so, which is the best way to speed up the sim/optim function? I greatly appreciate any help in this regard. Thank you in advance. Best, Carolina -- Carolina Plescia PhD candidate and IRCHSS Scholar Department of Political Science Trinity College Dublin College Green 2-3 Dublin 2 Email: plesciac(a)tcd.ie

12 years, 4 months

Problem with sim() when trying to use polynomial time in the random part within Zelig()

by Francois Maurice

Hi, The sim() fonction gives me the following message : Error in .local(object, ...) : Code not written yet The sim() object is not produce. I'm trying to evaluate a causal effect in a longitudinal study with a mixed model after mathing with MatchIt(). Here's the code : z.nearest <- zelig(outcome ~ treat + T + T2 + T3 + tag(1 + T + T2 | ID.factor), data=matched.nearest.long, model="ls.mixed")x.nearest.0 <- setx(z.nearest, fn=NULL, CINTER2G=0) x.nearest.1 <- setx(z.nearest, fn=NULL, CINTER2G=1) s.nearest <- sim(z.nearest, x=x.nearest.0, x1=x.nearest.1) I'm using polynomial time with a random part. There is no problem when the random part is only made of one term like this: z.nearest <- zelig(outcome ~ treat + T + T2 + T3 + tag(T | ID.factor), data=matched.nearest.long, model="ls.mixed") I posted a problem this last summer about polynomial time, but at that time the problem was within Zelig itself. Zelig was not able to cope with polynomial time, but that problem seems resolved. Maybe the problem with sim() has a link with that one. Thanks, Françcois Maurice

12 years, 4 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

Zelig January 2012