25 May
2010
25 May
'10
11:17 a.m.
Hi everyone,
I'm sorry to ask what might be a basic R question, but when I run:
impa.model <- zelig(impa_nm ~ age + age_sqr + hh_income_hundred + urban +
female, model = "logit.survey", weights = ~weight, ids = ~county, data =
my.data)
I get the warning:
In res$call <- as.call(zelig.call) :
Reached total allocation of 8175Mb: see help(memory.size)
After running summarize(impa.model) I get the output from the logit model,
but the setx commands I have written afterwards give the same above warning
message (see output below).
My computer has 8 G of RAM, so I can't allocate more memory I think. Even
though I have a large dataset (2.5 million observations), is it really
possible that the program requires so much memory?
Thank you kindly,
Prashant
#rest of code and output after the zelig command line
summary(impa.model)
Call:
zelig(formula = impa_nm ~ age + age_sqr + hh_income_hundred +
urban + female, model = "logit.survey", data = my.data,
weights = ~weight, ids = ~county)
Survey design:
svydesign(data = data, ids = ids, probs = probs, strata = strata,
fpc = fpc, nest = nest, check.strata = check.strata, weight = weights)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -4.227e+00 2.916e-02 -144.968 < 2e-16 ***
age 1.700e-02 1.129e-03 15.057 < 2e-16 ***
age_sqr 4.049e-04 1.187e-05 34.120 < 2e-16 ***
hh_income_hundred -2.853e-03 1.017e-04 -28.040 < 2e-16 ***
urban -7.631e-02 2.056e-02 -3.711 0.000222 ***
femalefemale -1.704e-01 8.307e-03 -20.513 < 2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 0.9801836)
Number of Fisher Scoring iterations: 7
WARNING INFORMATION
1: In class(y) <- oldClass(x) :
Reached total allocation of 8175Mb: see help(memory.size)
2: In class(y) <- oldClass(x) :
Reached total allocation of 8175Mb: see help(memory.size)
3: Reached total allocation of 8175Mb: see help(memory.size)
4: Reached total allocation of 8175Mb: see help(memory.size)
5: In ifelse(y > mu, d.res, -d.res) :
Reached total allocation of 8175Mb: see help(memory.size)
6: In ifelse(y > mu, d.res, -d.res) :
Reached total allocation of 8175Mb: see help(memory.size)
impair.urb15f.out <- setx(impa.model, urban = 1,
female = 1, age = 15,
hh_income_hundred = quantile(hh_income_hundred, .05:1))
impair.urb40f.out <- setx(impa.model, urban = 1,
female = 1, age = 40,
hh_income_hundred = quantile(hh_income_hundred, .05:1))
WARNING INFORMATION：
1: In as.list.data.frame(X) :
Reached total allocation of 8175Mb: see help(memory.size)
2: In as.list.data.frame(X) :
Reached total allocation of 8175Mb: see help(memory.size)
25 May
25 May
12:03 p.m.
If your dataset is very large, this could be a problem with Zelig.
However, if your data are relatively small (under a million
observations), then the problem probably isn't Zelig, but is probably
everything else that you have in your R workspace (which also eats up
memory, even if you're not using those objects at the moment).
Tips for freeing memory:
Try quitting R (don't save the workspace) and restarting the session.
If you don't want to quit, remove() extra data objects from the
workspace that you don't need. For example, if you cleaned your data
and created several versions of the data set, remove everything except
for my.data -- then type
gc()
Yours,
Olivia
On Tue, May 25, 2010 at 11:17 AM, Prashant <presearchwork(a)gmail.com> wrote:
Hi everyone,
I'm sorry to ask what might be a basic R question, but when I run:
impa.model <- zelig(impa_nm ~ age + age_sqr + hh_income_hundred + urban +
female, model = "logit.survey", weights = ~weight, ids = ~county, data =
my.data)
I get the warning:
In res$call <- as.call(zelig.call) :
Reached total allocation of 8175Mb: see help(memory.size)
After running summarize(impa.model) I get the output from the logit model,
but the setx commands I have written afterwards give the same above warning
message (see output below).
My computer has 8 G of RAM, so I can't allocate more memory I think. Even
though I have a large dataset (2.5 million observations), is it really
possible that the program requires so much memory?
Thank you kindly,
Prashant
#rest of code and output after the zelig command line
-
Zelig Mailing List, served by Harvard-MIT Data Center
Send messages: zelig(a)lists.gking.harvard.edu
[un]subscribe Options: http://lists.gking.harvard.edu/?info=zelig
Zelig program information: http://gking.harvard.edu/zelig/
summary(impa.model)
Call:
zelig(formula = impa_nm ~ age + age_sqr + hh_income_hundred +
urban + female, model = "logit.survey", data = my.data,
weights = ~weight, ids = ~county)
Survey design:
svydesign(data = data, ids = ids, probs = probs, strata = strata,
fpc = fpc, nest = nest, check.strata = check.strata, weight = weights)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -4.227e+00 2.916e-02 -144.968 < 2e-16 ***
age 1.700e-02 1.129e-03 15.057 < 2e-16 ***
age_sqr 4.049e-04 1.187e-05 34.120 < 2e-16 ***
hh_income_hundred -2.853e-03 1.017e-04 -28.040 < 2e-16 ***
urban -7.631e-02 2.056e-02 -3.711 0.000222 ***
femalefemale -1.704e-01 8.307e-03 -20.513 < 2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 0.9801836)
Number of Fisher Scoring iterations: 7
WARNING INFORMATION
1: In class(y) <- oldClass(x) :
Reached total allocation of 8175Mb: see help(memory.size)
2: In class(y) <- oldClass(x) :
Reached total allocation of 8175Mb: see help(memory.size)
3: Reached total allocation of 8175Mb: see help(memory.size)
4: Reached total allocation of 8175Mb: see help(memory.size)
5: In ifelse(y > mu, d.res, -d.res) :
Reached total allocation of 8175Mb: see help(memory.size)
6: In ifelse(y > mu, d.res, -d.res) :
Reached total allocation of 8175Mb: see help(memory.size)
impair.urb15f.out <- setx(impa.model, urban =
1, female = 1, age = 15,
hh_income_hundred = quantile(hh_income_hundred, .05:1))
impair.urb40f.out <- setx(impa.model, urban = 1, female = 1, age = 40,
hh_income_hundred = quantile(hh_income_hundred, .05:1))
WARNING INFORMATION：
1: In as.list.data.frame(X) :
Reached total allocation of 8175Mb: see help(memory.size)
2: In as.list.data.frame(X) :
Reached total allocation of 8175Mb: see help(memory.size) 
1:51 p.m.
Using the 64bit machine (and 64bit R) should help you too.
Kosuke
--
Department of Politics
Princeton University
http://imai.princeton.edu
On Tue, 25 May 2010, Olivia Lau wrote:
If your dataset is very large, this could be a problem
with Zelig.
However, if your data are relatively small (under a million
observations), then the problem probably isn't Zelig, but is probably
everything else that you have in your R workspace (which also eats up
memory, even if you're not using those objects at the moment).
Tips for freeing memory:
Try quitting R (don't save the workspace) and restarting the session.
If you don't want to quit, remove() extra data objects from the
workspace that you don't need. For example, if you cleaned your data
and created several versions of the data set, remove everything except
for my.data -- then type
gc()
Yours,
Olivia
On Tue, May 25, 2010 at 11:17 AM, Prashant <presearchwork(a)gmail.com> wrote:
Hi everyone,
I'm sorry to ask what might be a basic R question, but when I run:
impa.model <- zelig(impa_nm ~ age + age_sqr + hh_income_hundred + urban +
female, model = "logit.survey", weights = ~weight, ids = ~county, data =
my.data)
I get the warning:
In res$call <- as.call(zelig.call) :
Reached total allocation of 8175Mb: see help(memory.size)
After running summarize(impa.model) I get the output from the logit model,
but the setx commands I have written afterwards give the same above warning
message (see output below).
My computer has 8 G of RAM, so I can't allocate more memory I think. Even
though I have a large dataset (2.5 million observations), is it really
possible that the program requires so much memory?
Thank you kindly,
Prashant
#rest of code and output after the zelig command line
-
Zelig Mailing List, served by Harvard-MIT Data Center
Send messages: zelig(a)lists.gking.harvard.edu
[un]subscribe Options: http://lists.gking.harvard.edu/?info=zelig
Zelig program information: http://gking.harvard.edu/zelig/
summary(impa.model)
Call:
zelig(formula = impa_nm ~ age + age_sqr + hh_income_hundred +
urban + female, model = "logit.survey", data = my.data,
weights = ~weight, ids = ~county)
Survey design:
svydesign(data = data, ids = ids, probs = probs, strata = strata,
fpc = fpc, nest = nest, check.strata = check.strata, weight = weights)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -4.227e+00 2.916e-02 -144.968 < 2e-16 ***
age 1.700e-02 1.129e-03 15.057 < 2e-16 ***
age_sqr 4.049e-04 1.187e-05 34.120 < 2e-16 ***
hh_income_hundred -2.853e-03 1.017e-04 -28.040 < 2e-16 ***
urban -7.631e-02 2.056e-02 -3.711 0.000222 ***
femalefemale -1.704e-01 8.307e-03 -20.513 < 2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 0.9801836)
Number of Fisher Scoring iterations: 7
WARNING INFORMATION
1: In class(y) <- oldClass(x) :
Reached total allocation of 8175Mb: see help(memory.size)
2: In class(y) <- oldClass(x) :
Reached total allocation of 8175Mb: see help(memory.size)
3: Reached total allocation of 8175Mb: see help(memory.size)
4: Reached total allocation of 8175Mb: see help(memory.size)
5: In ifelse(y > mu, d.res, -d.res) :
Reached total allocation of 8175Mb: see help(memory.size)
6: In ifelse(y > mu, d.res, -d.res) :
Reached total allocation of 8175Mb: see help(memory.size)
impair.urb15f.out <- setx(impa.model, urban =
1, female = 1, age = 15,
hh_income_hundred = quantile(hh_income_hundred, .05:1))
impair.urb40f.out <- setx(impa.model, urban = 1, female = 1, age = 40,
hh_income_hundred = quantile(hh_income_hundred, .05:1))
WARNING INFORMATION：
1: In as.list.data.frame(X) :
Reached total allocation of 8175Mb: see help(memory.size)
2: In as.list.data.frame(X) :
Reached total allocation of 8175Mb: see help(memory.size)
5108
days inactive
5108
days old
2 comments
3 participants
participants (3)
-
Kosuke Imai -
Olivia Lau -
Prashant