Algoritmos Genéticos x Força Bruta para seleção de modelos de regressão

Author

Rodrigo Hermont Ozon

Published

September 11, 2023

Code 
start_time <- Sys.time()
Algoritmos Genéticos x Força Bruta para seleção de modelos

O cerne de qualquer modelo de regressão ou classificação reside na sua capacidade de elucidar a relação entre uma variável dependente e um conjunto de variáveis independentes, frequentemente referidas como preditores. A seleção desses preditores é uma tarefa desafiadora, pois não é intuitivamente evidente quais são essenciais e em que quantidade devem ser incorporados ao modelo.

A relevância dessa seleção é amplificada pelo risco de overfitting — quando um modelo é excessivamente complexo, capturando o ruído ao invés das verdadeiras relações subjacentes nos dados. Inversamente, um modelo com poucos preditores pode sofrer de underfitting, falhando em capturar relações significativas e comprometendo o seu desempenho preditivo.

Diante desses desafios, o objetivo torna-se identificar o modelo ideal que emprega um conjunto ótimo de preditores — aqueles que maximizam a explicação da variável dependente sem sucumbir ao ruído dos dados. Este artigo explora o uso de algoritmos genéticos como uma ferramenta eficaz para alcançar tal feito, destacando suas vantagens em relação às técnicas de força bruta, que, embora precisas, são computacionalmente onerosas e muitas vezes impraticáveis. Demonstramos como os algoritmos genéticos podem ser uma estratégia superior para equilibrar a complexidade do modelo e a precisão preditiva, conduzindo a insights mais profundos e a modelos mais generalizáveis.

Intro

A seleção de características é um passo crítico no processo de modelagem de regressão e classificação, pois influencia diretamente a performance e a eficiência do modelo final. Em contextos onde o número de características potenciais é vasto, a escolha de um método de seleção torna-se crucial. Tradicionalmente, poderíamos considerar a abordagem de força bruta, que testa todas as combinações possíveis de características para determinar o melhor conjunto. No entanto, essa estratégia é muitas vezes impraticável devido à sua complexidade exponencial, especialmente à medida que o número de características cresce. É aqui que os algoritmos genéticos (AGs) oferecem uma alternativa promissora.

Os algoritmos genéticos são inspirados nos processos de seleção natural e evolução biológica. Eles operam através da geração de uma população de soluções possíveis e, em seguida, aplicam operadores genéticos como seleção, cruzamento e mutação para evoluir as soluções ao longo de várias gerações. O objetivo é encontrar uma solução ótima ou satisfatória para o problema em questão.

Quando aplicados à seleção de características, os AGs têm várias vantagens sobre a abordagem de força bruta:

  • Eficiência Computacional: Ao invés de avaliar todas as combinações possíveis, os AGs exploram o espaço de busca de maneira inteligente, focando nas soluções mais promissoras. Isso pode reduzir significativamente o tempo de computação necessário.

  • Flexibilidade: Os AGs são capazes de lidar com espaços de busca não-lineares e com múltiplos ótimos locais, o que é comum em problemas de seleção de características.

  • Escalabilidade: Eles são mais escaláveis em relação ao número de características, tornando-os adequados para datasets de alta dimensão.

  • Adaptabilidade: Os AGs podem se adaptar a mudanças no espaço de busca, o que é útil em cenários onde os dados estão evoluindo.

  • Soluções Globais: Enquanto a força bruta garante encontrar a solução ótima global, ela é muitas vezes inviável. Os AGs, embora não garantam a solução ótima global, têm uma boa chance de se aproximar dela ou encontrar soluções que sejam suficientemente boas.

Dentro do contexto da seleção de variáveis, os algoritmos genéticos desempenham um papel crucial na mitigação do sobreajuste, ou overfitting, um problema comum quando um modelo é excessivamente complexo.

Fonte: Minhas (2021)

Ao simular o processo de evolução natural para otimizar a seleção de características, os algoritmos genéticos buscam um equilíbrio entre a adequação do modelo aos dados e a sua generalização para dados não vistos. Eles fazem isso ao penalizar conjuntos de características que são muito grandes ou que não contribuem significativamente para o poder preditivo do modelo. Isso é comparável à seleção natural, onde apenas os traços mais adaptáveis são passados para a próxima geração. Ao aplicar essa abordagem iterativa e seletiva, os algoritmos genéticos evitam eficazmente a armadilha de modelar o ruído estatístico presente nos dados de treinamento, o que é uma causa comum de sobreajuste em modelos preditivos. Assim, eles ajudam a assegurar que o modelo final seja robusto e confiável, com uma capacidade aprimorada de generalizar bem para novos conjuntos de dados.

Em resumo, os algoritmos genéticos oferecem uma abordagem mais viável e eficiente para a seleção de características em modelos de regressão e classificação, especialmente quando confrontados com um grande número de características. Eles não apenas economizam recursos computacionais valiosos, mas também fornecem um meio robusto e adaptável para navegar pelo complexo processo de identificar as características mais significativas para a modelagem.

Code 
library(dplyr)
library(tidyverse)
library(car)        
library(MASS)       
library(ISLR)       
library(tictoc)     
library(sjPlot)     
library(glmulti)    
library(flextable)  
library(performance)

theme_set(theme_light(base_size = 12))  # Ajusta os temas dos gráficos
theme_update(panel.grid.minor = element_blank())

Método StepWise e Força Bruta

A técnica stepwise é um método estatístico iterativo utilizado para selecionar um subconjunto ótimo de variáveis preditoras para um modelo estatístico, como regressão linear ou logística. Funciona através de um processo de seleção sequencial onde, em cada etapa, uma variável é adicionada ou removida do modelo baseado em critérios específicos, como o valor do teste F, o critério de informação de Akaike (AIC) ou o critério de informação bayesiano (BIC). Existem duas formas principais da técnica stepwise: a seleção para frente (forward selection), que começa com nenhum preditor e adiciona o mais significativo em cada passo, e a seleção para trás (backward elimination), que começa com todos os possíveis preditores e remove o menos significativo em cada etapa.

Fonte: Siddiqi et. alli (2022)

O processo continua até que nenhum novo preditor melhore significativamente o modelo ao ser adicionado ou removido, resultando em um modelo que se espera que tenha um bom equilíbrio entre a simplicidade e a capacidade preditiva.

Para exemplificar, utilizaremos aqui um conjunto de dados bastante conhecido:

Code 
glimpse(mtcars)
Rows: 32
Columns: 11
$ mpg  <dbl> 21.0, 21.0, 22.8, 21.4, 18.7, 18.1, 14.3, 24.4, 22.8, 19.2, 17.8,…
$ cyl  <dbl> 6, 6, 4, 6, 8, 6, 8, 4, 4, 6, 6, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 8,…
$ disp <dbl> 160.0, 160.0, 108.0, 258.0, 360.0, 225.0, 360.0, 146.7, 140.8, 16…
$ hp   <dbl> 110, 110, 93, 110, 175, 105, 245, 62, 95, 123, 123, 180, 180, 180…
$ drat <dbl> 3.90, 3.90, 3.85, 3.08, 3.15, 2.76, 3.21, 3.69, 3.92, 3.92, 3.92,…
$ wt   <dbl> 2.620, 2.875, 2.320, 3.215, 3.440, 3.460, 3.570, 3.190, 3.150, 3.…
$ qsec <dbl> 16.46, 17.02, 18.61, 19.44, 17.02, 20.22, 15.84, 20.00, 22.90, 18…
$ vs   <dbl> 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0,…
$ am   <dbl> 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0,…
$ gear <dbl> 4, 4, 4, 3, 3, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 4, 4, 4, 3, 3,…
$ carb <dbl> 4, 4, 1, 1, 2, 1, 4, 2, 2, 4, 4, 3, 3, 3, 4, 4, 4, 1, 2, 1, 1, 2,…
Code 
head(mtcars)

Se quisermos prever o consumo de combustível desses modelos de automóveis, (em mpg/milhas por galão) deveremos utilizar as combinações ideias das variáveis explicativas que melhor descreveriam tais relações:

  • Set de modelos (completo e nulo)
Code 
# Modelo completo e nulo
full_model <- glm(mpg ~ (hp + drat + wt + qsec + gear)^2, 
                 data = mtcars, family = gaussian)

null_model <- glm(mpg ~ 1, data = mtcars, family = gaussian)
  • Rodo o procedimento backward
Code 
# Roda o procedimento backward
optimal_model_backward <- step(full_model, direction = "backward",
                        scope = list(upper = full_model, lower = null_model))
  • Rodo o forward
Code 
optimal_model_forward <- step(null_model, direction = "forward",
                        scope = list(upper = full_model, lower = null_model))

  • Comparando o modelo escolhido pelo backward com o forward:

  • Melhor backward

Code 
summary(optimal_model_backward)

Call:
glm(formula = mpg ~ hp + drat + wt + qsec + gear + hp:drat + 
    hp:wt + hp:qsec + hp:gear + drat:wt + drat:qsec + wt:qsec + 
    wt:gear, family = gaussian, data = mtcars)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-2.51094  -0.94114  -0.04562   1.05155   2.50569  

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) -3.888e+02  1.425e+02  -2.728 0.013799 *  
hp           5.447e-01  2.178e-01   2.501 0.022258 *  
drat         2.668e+01  1.913e+01   1.395 0.180120    
wt           8.225e+01  2.679e+01   3.070 0.006592 ** 
qsec         1.713e+01  6.236e+00   2.748 0.013234 *  
gear         2.316e+01  5.420e+00   4.273 0.000457 ***
hp:drat     -1.236e-01  4.469e-02  -2.765 0.012748 *  
hp:wt       -5.294e-02  2.488e-02  -2.128 0.047392 *  
hp:qsec     -1.135e-02  6.886e-03  -1.649 0.116531    
hp:gear      5.046e-02  2.181e-02   2.314 0.032709 *  
drat:wt      5.970e+00  2.452e+00   2.435 0.025530 *  
drat:qsec   -1.422e+00  9.779e-01  -1.454 0.163231    
wt:qsec     -3.362e+00  1.082e+00  -3.107 0.006091 ** 
wt:gear     -9.950e+00  2.868e+00  -3.469 0.002738 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for gaussian family taken to be 2.851113)

    Null deviance: 1126.05  on 31  degrees of freedom
Residual deviance:   51.32  on 18  degrees of freedom
AIC: 135.93

Number of Fisher Scoring iterations: 2
  • Melhor forward
Code 
summary(optimal_model_forward)

Call:
glm(formula = mpg ~ wt + hp + qsec + gear + wt:hp, family = gaussian, 
    data = mtcars)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.9608  -1.2725  -0.5094   1.5570   4.3383  

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 33.998432   8.706505   3.905 0.000599 ***
wt          -7.983387   1.356531  -5.885 3.31e-06 ***
hp          -0.107998   0.025782  -4.189 0.000285 ***
qsec         0.555013   0.355563   1.561 0.130629    
gear         0.997389   0.691065   1.443 0.160887    
wt:hp        0.027265   0.007165   3.805 0.000775 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for gaussian family taken to be 4.310122)

    Null deviance: 1126.05  on 31  degrees of freedom
Residual deviance:  112.06  on 26  degrees of freedom
AIC: 144.92

Number of Fisher Scoring iterations: 2

A seleção de variáveis utilizando o método stepwise é uma técnica amplamente adotada na busca pelo modelo mais eficaz. Contudo, apesar de sua popularidade, ela não é infalível. O método stepwise incorpora duas estratégias principais: a seleção progressiva (forward selection) e a seleção regressiva (backward elimination). No entanto, essa abordagem enfrenta desafios significativos. Um deles é a falta de convergência para um modelo consistente, pois as estratégias progressiva e regressiva podem resultar em modelos distintos, como ilustrado em nosso exemplo. Além disso, mesmo na eventual convergência para um modelo comum, não há garantia de que este seja o mais adequado. Essa limitação decorre da natureza incremental do método stepwise, que avalia os modelos adicionando ou removendo variáveis sequencialmente, sem considerar todas as combinações possíveis de variáveis simultaneamente. Este processo iterativo de comparação e seleção pode, portanto, negligenciar a configuração ideal de variáveis que maximizaria o desempenho do modelo.

Veja pelo teste de \(\chi^2\) na ANOVA e da tabela de métricas de performance:

  • \(\chi^2\)
Code 
anova(optimal_model_backward, optimal_model_forward, test = "Chisq")
  • Tabela comparativa:
Code 
compare_performance(optimal_model_backward, optimal_model_forward)

Aqui os critérios de informação (AIC, BIC) são preferíveis em relação ao R^2 por exemplo, pois eles demonstram melhor o ajuste do modelo, onde ele possa ser analisado/considerado como aquele de menor AIC/BIC como preferível, na medida em que o modelo for incorporando mais preditores (mais parâmetros então maior a chance de overfitting, p. ex. e menor, underfitting).

Utilizando o pacote glmulti no ambiente de programação R, é viável implementar um método exaustivo que constrói todos os modelos estatísticos concebíveis, levando em conta todas as combinações de preditores e, se desejado, suas interações binárias. Esta técnica é conhecida como “força bruta”, refletindo sua abrangência e intensidade computacional.

O glmulti procede à avaliação comparativa dos modelos, baseando-se na quantidade de informação relevante que cada um oferece. Para tal, emprega-se critérios de informação, tais como o Critério de Informação de Akaike (AIC) ou o Critério de Informação Bayesiano (BIC). Esses critérios são preferidos em detrimento de métricas tradicionais como o R², pois oferecem uma medida de “adequação” do modelo que incorpora uma penalidade proporcional ao número de preditores utilizados.

Fonte: Hebbali (2020)

Diferentemente dos critérios de informação, o R² tende a aumentar à medida que mais termos são adicionados ao modelo, o que pode levar a um sobreajuste. Um modelo sobreajustado é problemático, pois tende a refletir o ruído dos dados em vez de capturar as relações substanciais entre as variáveis. Isso torna os coeficientes estimados e os valores-p associados pouco confiáveis para inferências estatísticas.

O sobreajuste não apenas compromete a precisão do modelo, mas também sua aplicabilidade, pois um modelo que se ajusta demais às idiossincrasias de uma amostra específica falha em ser generalizável para outras amostras, reduzindo sua utilidade prática. Portanto, é essencial construir uma gama abrangente de modelos e utilizar Critérios de Informação para sua comparação, em vez de se fiar exclusivamente no R². Apesar da eficácia da abordagem de força bruta, o desafio reside na gestão do grande número de modelos potenciais, o que exige estratégias para otimizar o processo de seleção e análise, mantendo a integridade e a eficiência computacional.

Mudamos o nosso dataset (não mais o mtcars) e neste estudo, nosso foco é analisar o salário de 3.000 trabalhadores americanos, utilizando cinco variáveis preditoras: classe de trabalho, educação, idade, saúde e seguro de saúde. Para realizar essa análise, selecionamos o conjunto de dados “Wage”, disponível no pacote ISLR do ambiente estatístico R.

A escolha do Critério de Informação (CI) para avaliar os modelos é crucial. Por padrão, utilizamos o Critério de Informação de Akaike (AIC), mas temos à disposição outras opções como o Critério de Informação Bayesiano (BIC), o quasi-AIC para dados com superdispersão ou de contagem (qaic e qaicc), e o AIC corrigido para amostras pequenas (aicc). Este último é de minha preferência pessoal, pois oferece resultados consistentes com o AIC em amostras grandes e supera seu desempenho em amostras menores, proporcionando uma avaliação mais precisa em contextos com limitações de dados. Essa flexibilidade na escolha do CI permite uma adaptação mais fina à natureza dos dados e aos objetivos específicos da análise, garantindo que a seleção do modelo seja tanto rigorosa quanto relevante para a interpretação dos fatores que influenciam os salários no mercado de trabalho americano.

Veremos até quantos candidatos a modelos possíveis rodando a rotina presente no pacote glmulti. O argumento method = "d" conta o número de modelos candidatos sem realizar nenhum cálculo. Para o nosso exemplo com 5 preditores, teremos 32 modelos sem interações e 1921 modelos com interações. Se o method = "h", uma triagem exaustiva é realizada, o que significa que todos os modelos possíveis serão criados. Se o method = "g", o algoritmo genético é empregado (recomendado para grandes conjuntos de candidatos).

  • Possibilidades de Modelos sem as interações
Code 
glmulti(wage   ~ jobclass + education + age + health + health_ins,
        data   = Wage, 
        crit   = aicc,       # AICC é o AIC corrigido para pequenas amostras
        level  = 1,          # 2 com interações, 1 sem  
        method = "d",        # "d", ou "h", ou "g"
        family = gaussian, 
        fitfunction = glm,   # Tipo de modelo (LM, GLM etc.)
        confsetsize = 100)   # utiliza somente os 100 melhores modelos
Initialization...
TASK: Diagnostic of candidate set.
Sample size: 3000
4 factor(s).
1 covariate(s).
0 f exclusion(s).
0 c exclusion(s).
0 f:f exclusion(s).
0 c:c exclusion(s).
0 f:c exclusion(s).
Size constraints: min =  0 max = -1
Complexity constraints: min =  0 max = -1
Your candidate set contains 32 models.
[1] 32
  • Possibilidades de Modelos com as interações
Code 
glmulti(wage   ~ jobclass + education + age + health + health_ins,
        data   = Wage, 
        crit   = aicc,       # AICC é o AIC corrigido para pequenas amostras
        level  = 2,          # 2 com interações, 1 sem  
        method = "d",        # "d", ou "h", ou "g"
        family = gaussian, 
        fitfunction = glm,   # Tipo de modelo (LM, GLM etc.)
        confsetsize = 100)   # utiliza somente os 100 melhores modelos
Initialization...
TASK: Diagnostic of candidate set.
Sample size: 3000
4 factor(s).
1 covariate(s).
0 f exclusion(s).
0 c exclusion(s).
0 f:f exclusion(s).
0 c:c exclusion(s).
0 f:c exclusion(s).
Size constraints: min =  0 max = -1
Complexity constraints: min =  0 max = -1
Your candidate set contains 1921 models.
[1] 1921

Veja que dispomos de 32 modelos válidos sem interações entre as explicativas e 1921 com as interações entre elas.

Procederemos agora à execução do algoritmo exaustivo para calcular 1921 regressões, com o objetivo de identificar o modelo ótimo que integra as interações entre os cinco preditores selecionados. Para monitorar o tempo de processamento, utilizaremos as funções de contagem de tempo do pacote tictoc.

De maneira encorajadora, o método exaustivo consumiu meros 60 segundos para ser concluído. Esta é uma performance notável, a meu ver. Contudo, é importante salientar que, em situações onde o número de preditores excede significativamente cinco, podemos nos deparar com desafios relacionados ao desempenho computacional.

Code 
tic()

h_model <- glmulti(wage ~ jobclass + education + age + health + health_ins,
          data   = Wage, 
          crit   = aicc,       
          level  = 2,         
          method = "h",       
          family = gaussian, 
          fitfunction = glm,   
          confsetsize = 100)  
Initialization...
TASK: Exhaustive screening of candidate set.
Fitting...

After 50 models:
Best model: wage~1+education+age+jobclass:age+health:age+health_ins:age
Crit= 29827.864295308
Mean crit= 30184.7335535017


After 100 models:
Best model: wage~1+jobclass+education+age+education:jobclass+education:age+health:age+health_ins:age
Crit= 29815.9273144544
Mean crit= 30114.5541058805


After 150 models:
Best model: wage~1+jobclass+education+age+education:jobclass+education:age+health:age+health_ins:age
Crit= 29815.9273144544
Mean crit= 29934.7534975215


After 200 models:
Best model: wage~1+jobclass+education+age+education:jobclass+education:age+health:age+health_ins:age
Crit= 29815.9273144544
Mean crit= 29885.0804095925


After 250 models:
Best model: wage~1+jobclass+education+age+education:jobclass+education:age+health:age+health_ins:age
Crit= 29815.9273144544
Mean crit= 29853.1589383865


After 300 models:
Best model: wage~1+jobclass+education+health+age+education:jobclass+education:age+health:age+health_ins:age
Crit= 29815.7251202986
Mean crit= 29834.13579651


After 350 models:
Best model: wage~1+jobclass+education+health+age+education:jobclass+education:age+health:age+health_ins:age
Crit= 29815.7251202986
Mean crit= 29829.7529704456


After 400 models:
Best model: wage~1+education+health_ins+age+health_ins:education+education:age+health:age
Crit= 29805.2622191833
Mean crit= 29825.4446546592


After 450 models:
Best model: wage~1+education+health_ins+age+health_ins:education+jobclass:age+education:age+health:age+health_ins:age
Crit= 29803.0289240545
Mean crit= 29820.4943030812


After 500 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+education:age+health:age
Crit= 29796.7544660965
Mean crit= 29814.2488044282


After 550 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29808.3093463257

After 600 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29808.3093463257


After 650 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29808.3093463257

After 700 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29808.3093463257

After 750 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29806.9777994778


After 800 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29805.3845834103


After 850 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29805.2590912908


After 900 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29805.2237304162


After 950 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29805.0505615358


After 1000 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29804.3137547809


After 1050 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29804.0902716277


After 1100 models:
Best model: wage~1+jobclass+education+health_ins+age+education:jobclass+health_ins:education+education:age+health:age
Crit= 29794.403562754
Mean crit= 29802.3073891049


After 1150 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29800.5809471328


After 1200 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29799.7275115013


After 1250 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29799.7256681841

After 1300 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29799.7256681841


After 1350 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29799.5210963766


After 1400 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.9406807532


After 1450 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.7409828125


After 1500 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6895787015

After 1550 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6895787015


After 1600 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6818967865


After 1650 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6284501407


After 1700 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6265738983


After 1750 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6253233475

After 1800 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6253233475


After 1850 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6253233475

After 1900 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6232989334

After 1950 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6232989334


After 2000 models:
Best model: wage~1+jobclass+education+health+health_ins+age+education:jobclass+health_ins:education+education:age+health:age+health_ins:age
Crit= 29793.4306133546
Mean crit= 29798.6232989334
Completed.
Code 
toc() # 19 sec elapsed: 1921 models 
24.23 sec elapsed

Técnicas de melhoria de desempenho

A estratégia inicial para aprimorar o desempenho consiste na eliminação de preditores ou interações que sejam redundantes. Tomemos, por exemplo, o peso e o índice de massa corporal (IMC), que tendem a fornecer informações sobrepostas — em termos estatísticos, diríamos que apresentam alta multicolinearidade. A inclusão simultânea de ambos os termos pode inflar desnecessariamente a quantidade de modelos possíveis, sem agregar valor significativo à análise.

Ilustrativamente, a adição de apenas dois preditores categóricos adicionais — estado civil (maritl) e região (region) — ao modelo salarial previamente mencionado, eleva o número de modelos potenciais para mais de 2,5 milhões (exatamente 2.604.485).

Code 
glmulti(wage ~ jobclass + education + age + health + health_ins + maritl + region,
        data   = Wage, 
        crit   = aicc,       
        level  = 2,           
        method = "d",        
        family = gaussian, 
        fitfunction = glm,  
        confsetsize = 100,
        plotty=FALSE)
Initialization...
TASK: Diagnostic of candidate set.
Sample size: 3000
6 factor(s).
1 covariate(s).
0 f exclusion(s).
0 c exclusion(s).
0 f:f exclusion(s).
0 c:c exclusion(s).
0 f:c exclusion(s).
Size constraints: min =  0 max = -1
Complexity constraints: min =  0 max = -1
Your candidate set contains 2604485 models.
[1] 2604485

Diante de um volume tão elevado, que excede a capacidade de processamento convencional, o algoritmo genético surge como uma solução viável, capaz de navegar por este vasto espaço de modelos de maneira eficiente.

Ao lidar com um conjunto de 6 preditores numéricos e suas respectivas interações, a metodologia de “força bruta” pode demandar aproximadamente 3 horas para processamento completo. [Aqui demonstraremos somente com 5 preditores, para fins de comparação com o tempo e resultados obtidos do método exaustivo x força bruta e o tempo de execução.] Em contraste, a utilização de algoritmos genéticos reduz significativamente esse tempo para apenas 50 a 60 segundos. Essa abordagem eficiente não só economiza tempo valioso, mas também alcança resultados comparáveis aos obtidos pelo método mais demorado, embora possa ocasionalmente resultar em valores ligeiramente superiores do Critério de Informação.

Vamos comparar os timings com o nosso primeiro dataset de exemplo, o mtcars:

Método Exaustivo

Code 
tic()

test_h <- glmulti(mpg ~ hp + drat + wt + qsec + gear, 
                 data   = mtcars, 
                 method = "h",       # h = método exaustivo (força bruta)
                 crit   = aic,     
                 level  = 2,        
                 family = gaussian,
                 fitfunction = glm, 
                 confsetsize = 100)  
Initialization...
TASK: Exhaustive screening of candidate set.
Fitting...

After 50 models:
Best model: mpg~1+gear:wt+gear:qsec
Crit= 146.193186152864
Mean crit= 168.372599842419


After 100 models:
Best model: mpg~1+gear:wt+gear:qsec
Crit= 146.193186152864
Mean crit= 163.894646890807


After 150 models:
Best model: mpg~1+drat:hp+wt:hp+qsec:wt+gear:wt+gear:qsec
Crit= 144.598329266555
Mean crit= 156.275404945991


After 200 models:
Best model: mpg~1+drat:hp+wt:hp+qsec:wt+gear:wt+gear:qsec
Crit= 144.598329266555
Mean crit= 152.132216200647


After 250 models:
Best model: mpg~1+drat:hp+wt:hp+qsec:wt+gear:wt+gear:qsec
Crit= 144.598329266555
Mean crit= 151.033663189397


After 300 models:
Best model: mpg~1+drat:hp+wt:hp+qsec:wt+gear:wt+gear:qsec
Crit= 144.598329266555
Mean crit= 149.891456335748


After 350 models:
Best model: mpg~1+drat:hp+wt:hp+qsec:wt+gear:wt+gear:qsec
Crit= 144.598329266555
Mean crit= 149.767066238731


After 400 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 148.046431384816


After 450 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 148.016745111225


After 500 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 147.3045629641


After 550 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 146.774135351077


After 600 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 146.74082023953


After 650 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 146.655123900789


After 700 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 146.421621806416


After 750 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 146.39399961777


After 800 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 146.225695114579

After 850 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 146.225695114579


After 900 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 145.723671137625


After 950 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 145.410493274487


After 1000 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 145.397542227074


After 1050 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 145.071831444856


After 1100 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.950969832488


After 1150 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.903156753665


After 1200 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.65302186714

After 1250 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.65302186714


After 1300 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.592234777517


After 1350 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.589177719307

After 1400 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.589177719307


After 1450 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.248652513689


After 1500 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.174066231571


After 1550 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 144.060744608176


After 1600 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.957703689905

After 1650 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.957703689905


After 1700 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.911391196911


After 1750 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.910062639886

After 1800 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.910062639886


After 1850 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.907721129938

After 1900 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.907721129938


After 1950 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.850335944671


After 2000 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.831959182338


After 2050 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.813033700993


After 2100 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.807150141774


After 2150 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.793065937144

After 2200 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.793065937144


After 2250 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.793065937144

After 2300 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.793065937144

After 2350 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.793065937144

After 2400 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.793065937144

After 2450 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.793065937144

After 2500 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.663029986256


After 2550 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.630240069528

After 2600 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.630240069528


After 2650 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.566819498536

After 2700 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.566819498536


After 2750 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.565794937649

After 2800 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.565794937649


After 2850 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.565794937649

After 2900 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.559292531709

After 2950 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.559292531709


After 3000 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.452353832058


After 3050 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.385037645174

After 3100 models:
Best model: mpg~1+wt:hp+qsec:hp+gear:hp+gear:wt+gear:qsec
Crit= 140.628643020179
Mean crit= 143.385037645174


After 3150 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.260867812097


After 3200 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.186966766267

After 3250 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.186966766267


After 3300 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.180419987129

After 3350 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.180419987129


After 3400 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.180419987129

After 3450 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.180419987129

After 3500 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.180419987129

After 3550 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.142157205844


After 3600 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.138804831927


After 3650 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.122923388803


After 3700 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.119058953281


After 3750 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.105234928175


After 3800 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.052421953374


After 3850 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.050087044238

After 3900 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.050087044238


After 3950 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.021973069064

After 4000 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.021973069064


After 4050 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.020454535585


After 4100 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.00935048557

After 4150 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 143.00935048557


After 4200 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.927490086569


After 4250 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.883277432069

After 4300 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.883277432069


After 4350 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.883277432069

After 4400 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.883277432069

After 4450 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.878468688408

After 4500 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.878468688408


After 4550 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.878468688408

After 4600 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.794683921545


After 4650 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.785615187281


After 4700 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.775126972353


After 4750 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.767167744989

After 4800 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.767167744989


After 4850 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.767167744989

After 4900 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.767167744989

After 4950 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.767167744989

After 5000 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.767167744989

After 5050 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.767167744989

After 5100 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.748767339728


After 5150 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.741904736048

After 5200 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.741904736048


After 5250 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.741904736048

After 5300 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.741904736048

After 5350 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.741904736048

After 5400 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.730003302893

After 5450 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.730003302893


After 5500 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.730003302893

After 5550 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.730003302893

After 5600 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.730003302893

After 5650 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.730003302893

After 5700 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.721224277867

After 5750 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.721224277867


After 5800 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.702272639738

After 5850 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.702272639738


After 5900 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066

After 5950 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066


After 6000 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066

After 6050 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066

After 6100 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066

After 6150 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066

After 6200 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066

After 6250 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.699394730066

After 6300 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6350 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012


After 6400 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6450 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6500 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6550 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6600 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6650 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6700 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.696888401012

After 6750 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.683897804617

After 6800 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.683897804617


After 6850 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.681092171117

After 6900 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.681092171117


After 6950 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.681092171117

After 7000 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.681092171117

After 7050 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.681092171117

After 7100 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.677646573869

After 7150 models:
Best model: mpg~1+drat+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:drat+gear:wt
Crit= 140.327972579494
Mean crit= 142.677646573869


After 7200 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.591531333506


After 7250 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.543518292473


After 7300 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.519136960098


After 7350 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.446550736014


After 7400 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7450 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103


After 7500 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7550 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7600 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7650 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7700 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7750 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7800 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7850 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7900 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 7950 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.40296677103

After 8000 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.383593084206

After 8050 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.383593084206


After 8100 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.383593084206

After 8150 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.383593084206

After 8200 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.383593084206

After 8250 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.365297487215

After 8300 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.365297487215


After 8350 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.365297487215

After 8400 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.333555459399


After 8450 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.324919477457


After 8500 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.316522212038

After 8550 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.316522212038


After 8600 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.31579997157

After 8650 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.31579997157


After 8700 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.31579997157

After 8750 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.31579997157

After 8800 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.31579997157

After 8850 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.286314238443


After 8900 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.282449744035

After 8950 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.282449744035


After 9000 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.282449744035

After 9050 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.282449744035

After 9100 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.282449744035

After 9150 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.282449744035

After 9200 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.282449744035

After 9250 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.279661734499

After 9300 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.279661734499


After 9350 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.279661734499

After 9400 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.273901408575

After 9450 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.273901408575


After 9500 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.273901408575

After 9550 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.273901408575

After 9600 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.273901408575

After 9650 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.266896067511

After 9700 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.266896067511


After 9750 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.266896067511

After 9800 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.266896067511

After 9850 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.266896067511

After 9900 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.26068799811

After 9950 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.26068799811


After 10000 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.26068799811

After 10050 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.26068799811

After 10100 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.26068799811

After 10150 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10200 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182


After 10250 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10300 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10350 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10400 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10450 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10500 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10550 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10600 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10650 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10700 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10750 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10800 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.258189170182

After 10850 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.243202319022

After 10900 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.243202319022


After 10950 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.238767933296


After 11000 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.235023222431

After 11050 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.235023222431


After 11100 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.235023222431

After 11150 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11200 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137


After 11250 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11300 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11350 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11400 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11450 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11500 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11550 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.226489439137

After 11600 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.220087815577

After 11650 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.220087815577


After 11700 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.220087815577

After 11750 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.215714498142

After 11800 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.215714498142


After 11850 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.215714498142

After 11900 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.215714498142

After 11950 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.215714498142

After 12000 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.215714498142

After 12050 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.215714498142

After 12100 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.204411681334

After 12150 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.204411681334


After 12200 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12250 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929


After 12300 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12350 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12400 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12450 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12500 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12550 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12600 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.186123338929

After 12650 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.183696154577

After 12700 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.183696154577


After 12750 models:
Best model: mpg~1+drat+wt+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 140.216347673733
Mean crit= 142.167665445974


After 12800 models:
Best model: mpg~1+wt+qsec+drat:hp+qsec:wt+gear:wt+gear:qsec
Crit= 137.962055577239
Mean crit= 142.012601685718


After 12850 models:
Best model: mpg~1+wt+qsec+drat:hp+qsec:wt+gear:wt+gear:qsec
Crit= 137.962055577239
Mean crit= 141.882060446346


After 12900 models:
Best model: mpg~1+wt+qsec+drat:hp+qsec:wt+gear:wt+gear:qsec
Crit= 137.962055577239
Mean crit= 141.847846257201


After 12950 models:
Best model: mpg~1+wt+qsec+drat:hp+qsec:wt+gear:wt+gear:qsec
Crit= 137.962055577239
Mean crit= 141.693364036808


After 13000 models:
Best model: mpg~1+wt+qsec+drat:hp+qsec:wt+gear:wt+gear:qsec
Crit= 137.962055577239
Mean crit= 141.629354319022


After 13050 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 141.39954051034


After 13100 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 141.32899562022


After 13150 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 141.177531587486


After 13200 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 141.123480833253


After 13250 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 141.067149204515


After 13300 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 141.027078910074


After 13350 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.893027766836


After 13400 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.849785808718


After 13450 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.753828160856


After 13500 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.749759458845


After 13550 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.602691671226


After 13600 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.539735802144


After 13650 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.518136349965


After 13700 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.465101194717


After 13750 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.444454146992


After 13800 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.382702533181


After 13850 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.299663846239


After 13900 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.260830678175


After 13950 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.20196828232


After 14000 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.177945244051


After 14050 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.176087506284


After 14100 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.136962041821


After 14150 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.132035280437


After 14200 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.120678200273

After 14250 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.120678200273


After 14300 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.120678200273

After 14350 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.056217297047


After 14400 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.037047137084

After 14450 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 140.037047137084


After 14500 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 139.961644009469


After 14550 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 139.949033140967


After 14600 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 139.923996715044

After 14650 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 139.923996715044


After 14700 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 139.923996715044

After 14750 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 139.895080277593


After 14800 models:
Best model: mpg~1+wt+qsec+qsec:hp+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.660228749827
Mean crit= 139.868759951041


After 14850 models:
Best model: mpg~1+hp+wt+qsec+drat:hp+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:drat+gear:wt+gear:qsec
Crit= 137.453368309053
Mean crit= 139.809760796542


After 14900 models:
Best model: mpg~1+hp+wt+qsec+drat:hp+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:drat+gear:wt+gear:qsec
Crit= 137.453368309053
Mean crit= 139.749790493431


After 14950 models:
Best model: mpg~1+hp+wt+qsec+drat:hp+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:drat+gear:wt+gear:qsec
Crit= 137.453368309053
Mean crit= 139.743616265377


After 15000 models:
Best model: mpg~1+hp+wt+qsec+drat:hp+wt:hp+wt:drat+qsec:hp+qsec:drat+qsec:wt+gear:hp+gear:drat+gear:wt+gear:qsec
Crit= 137.453368309053
Mean crit= 139.741374462876


After 15050 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.659900531309


After 15100 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.621970006849


After 15150 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.557049482478


After 15200 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.549525238796

After 15250 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.549525238796


After 15300 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.518644517935

After 15350 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.518644517935


After 15400 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.518644517935

After 15450 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.50229894768

After 15500 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.50229894768


After 15550 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.48582701636


After 15600 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.464321499355


After 15650 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.461975566982


After 15700 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.460740304343

After 15750 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.460740304343


After 15800 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.460740304343

After 15850 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.460740304343

After 15900 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.460740304343

After 15950 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.417636091552

After 16000 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.417636091552


After 16050 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.417636091552

After 16100 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.382069633976


After 16150 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.377054310835

After 16200 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.377054310835


After 16250 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.377054310835

After 16300 models:
Best model: mpg~1+drat+wt+qsec+drat:hp+wt:drat+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 137.063069274529
Mean crit= 139.377054310835

After 16350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.337397875779


After 16400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.316115097192


After 16450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.28270941934

After 16500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.28270941934


After 16550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.28270941934

After 16600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.28270941934

After 16650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.273843090753


After 16700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.245042860795

After 16750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.245042860795


After 16800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.245042860795

After 16850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.245042860795

After 16900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.22088262887


After 16950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886


After 17050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 17950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 18950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 19950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 20950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 21950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 22950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 23950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 24950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 25950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 26950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 27950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28750 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28800 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28850 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28900 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 28950 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29000 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29050 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29100 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29150 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29200 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29250 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29300 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29350 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29400 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29450 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29500 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29550 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29600 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29650 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.192634081886

After 29700 models:
Best model: mpg~1+hp+drat+wt+qsec+drat:hp+wt:drat+qsec:hp+qsec:wt+gear:hp+gear:drat+gear:wt
Crit= 136.432231613786
Mean crit= 139.144358723309


After 29750 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.844398514929


After 29800 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.725917255096


After 29850 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.647459750123


After 29900 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.584409864494


After 29950 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.464900649394


After 30000 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.382474914287


After 30050 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.34248844609


After 30100 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.301385261323


After 30150 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.286920567034

After 30200 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.286920567034


After 30250 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.221871909021


After 30300 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.187247620277

After 30350 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.187247620277


After 30400 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.162576582241

After 30450 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.162576582241


After 30500 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.111948064617


After 30550 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.106051423334


After 30600 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.100874896638

After 30650 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.100874896638


After 30700 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.100874896638

After 30750 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.049618745098


After 30800 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.010466534029


After 30850 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 138.001228185536


After 30900 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.984387044258


After 30950 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.984016489485


After 31000 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.965833258805


After 31050 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.954562410382

After 31100 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.954562410382


After 31150 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.954539025041

After 31200 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.954539025041


After 31250 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.865496121793


After 31300 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.846881896295

After 31350 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.846881896295


After 31400 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.846881896295

After 31450 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.846881896295

After 31500 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.846881896295

After 31550 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.844386727569

After 31600 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.844386727569


After 31650 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.844386727569

After 31700 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.84170241616

After 31750 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.84170241616


After 31800 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.770707604355


After 31850 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.749240780882


After 31900 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.73003912775

After 31950 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.73003912775


After 32000 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.719993780494

After 32050 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.719993780494


After 32100 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.689845720356


After 32150 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.688070612082


After 32200 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.686219347153

After 32250 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.686219347153


After 32300 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.686219347153

After 32350 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.686219347153

After 32400 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.685759146871

After 32450 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.685759146871


After 32500 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.685759146871

After 32550 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.685759146871

After 32600 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.683698807926

After 32650 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.683698807926


After 32700 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.683698807926

After 32750 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.683698807926

After 32800 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.683698807926

After 32850 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.667652413533

After 32900 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.667652413533


After 32950 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.667652413533

After 33000 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.667652413533

After 33050 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.667652413533

After 33100 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.667652413533

After 33150 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.665246841122

After 33200 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.665246841122


After 33250 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.665246841122

After 33300 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.659533084601


After 33350 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.649382297842


After 33400 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633798864632

After 33450 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633798864632


After 33500 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633798864632

After 33550 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633798864632

After 33600 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633798864632

After 33650 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633597280441

After 33700 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633597280441


After 33750 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633597280441

After 33800 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633597280441

After 33850 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.633597280441

After 33900 models:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 137.590733321479

Completed.
Code 
toc()
124.96 sec elapsed

A excelência do algoritmo genético é reconhecida, particularmente pela sua capacidade de processamento rápido. No entanto, a sua aplicabilidade universal não é isenta de limitações. Notavelmente, quando se trata de variáveis categóricas com um vasto número de categorias, a performance do algoritmo genético pode ser eclipsada pela abordagem exaustiva. A título de exemplo, o nosso modelo de análise salarial, que incorpora uma multiplicidade de preditores categóricos, completou a triagem exaustiva em apenas 19 segundos, enquanto o algoritmo genético demandou 117 segundos para alcançar a convergência, resultando em um tempo quase seis vezes superior. Adicionalmente, o algoritmo genético está sujeito a desafios de convergência, podendo operar por um período indefinido sem previsibilidade de conclusão. Mais ainda, o método exaustivo tende a resultar em valores de Critério de Informação superiores. Portanto, recomenda-se enfaticamente a geração de todos os modelos possíveis por meio da triagem exaustiva, ou da aplicação da metodologia de “força bruta”, sempre que factível, reservando o uso do algoritmo genético estritamente para contextos com uma quantidade substancial de preditores numéricos.

Algoritmos Genéticos

Code 
tic()

test_g <- glmulti(mpg ~ hp + drat + wt + qsec + gear, 
                 data   = mtcars, 
                 method = "g",     # g = genetic algorithms
                 crit   = aic,      
                 level  = 2,         
                 family = gaussian,
                 fitfunction = glm,  
                 confsetsize = 100) 
Initialization...
TASK: Genetic algorithm in the candidate set.
Initialization...
Algorithm started...

After 10 generations:
Best model: mpg~1+hp+wt+qsec+qsec:drat+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 138.975661358302
Mean crit= 151.182613166538

Change in best IC: -9861.0243386417 / Change in mean IC: -9848.81738683346

After 20 generations:
Best model: mpg~1+hp+wt+qsec+qsec:wt+gear:hp+gear:wt+gear:qsec
Crit= 137.952546857568
Mean crit= 149.035616345581

Change in best IC: -1.02311450073427 / Change in mean IC: -2.14699682095701

After 30 generations:
Best model: mpg~1+wt+qsec+gear+qsec:hp+qsec:wt+gear:drat+gear:wt
Crit= 136.61102526519
Mean crit= 147.398777170023

Change in best IC: -1.34152159237763 / Change in mean IC: -1.63683917555778

After 40 generations:
Best model: mpg~1+wt+qsec+gear+qsec:hp+qsec:wt+gear:drat+gear:wt
Crit= 136.61102526519
Mean crit= 146.494370090289

Change in best IC: 0 / Change in mean IC: -0.904407079734369

After 50 generations:
Best model: mpg~1+wt+qsec+gear+qsec:hp+qsec:wt+gear:wt
Crit= 136.237445139365
Mean crit= 145.721389786794

Change in best IC: -0.373580125824589 / Change in mean IC: -0.772980303494819

After 60 generations:
Best model: mpg~1+wt+qsec+gear+qsec:hp+qsec:wt+gear:wt
Crit= 136.237445139365
Mean crit= 145.383956733359

Change in best IC: 0 / Change in mean IC: -0.337433053434893

After 70 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 144.930314046032

Change in best IC: -1.50341598754119 / Change in mean IC: -0.453642687327317

After 80 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 144.494042151362

Change in best IC: 0 / Change in mean IC: -0.436271894669829

After 90 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 144.285616291053

Change in best IC: 0 / Change in mean IC: -0.208425860308608

After 100 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 144.086705632512

Change in best IC: 0 / Change in mean IC: -0.198910658541791

After 110 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 143.845619666556

Change in best IC: 0 / Change in mean IC: -0.241085965955108

After 120 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 143.769297341986

Change in best IC: 0 / Change in mean IC: -0.076322324570782

After 130 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 143.698759944009

Change in best IC: 0 / Change in mean IC: -0.0705373979762385

After 140 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 143.50296689591

Change in best IC: 0 / Change in mean IC: -0.195793048099262

After 150 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 143.307720319917

Change in best IC: 0 / Change in mean IC: -0.195246575993423

After 160 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 143.068987070956

Change in best IC: 0 / Change in mean IC: -0.238733248961154

After 170 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 142.739625495126

Change in best IC: 0 / Change in mean IC: -0.329361575829495

After 180 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 142.695489389972

Change in best IC: 0 / Change in mean IC: -0.0441361051541094

After 190 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 142.523711361974

Change in best IC: 0 / Change in mean IC: -0.171778027997533

After 200 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 142.462625149521

Change in best IC: 0 / Change in mean IC: -0.0610862124534037

After 210 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 142.375689894182

Change in best IC: 0 / Change in mean IC: -0.0869352553392844

After 220 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 142.183310509389

Change in best IC: 0 / Change in mean IC: -0.192379384793071

After 230 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 142.145646665506

Change in best IC: 0 / Change in mean IC: -0.037663843883081

After 240 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.919882392764

Change in best IC: 0 / Change in mean IC: -0.225764272741344

After 250 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.894974792564

Change in best IC: 0 / Change in mean IC: -0.0249076002006063

After 260 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.815171803777

Change in best IC: 0 / Change in mean IC: -0.0798029887868665

After 270 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.75312416651
Change in best IC: 0 / Change in mean IC: -0.0620476372663177

After 280 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.75312416651

Change in best IC: 0 / Change in mean IC: 0

After 290 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.642113671989

Change in best IC: 0 / Change in mean IC: -0.111010494521167

After 300 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.615809116402

Change in best IC: 0 / Change in mean IC: -0.026304555586961

After 310 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.611199773051

Change in best IC: 0 / Change in mean IC: -0.00460934335146135

After 320 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.545242722748

Change in best IC: 0 / Change in mean IC: -0.0659570503032967

After 330 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.418729154527

Change in best IC: 0 / Change in mean IC: -0.126513568220304

After 340 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.380288866522

Change in best IC: 0 / Change in mean IC: -0.0384402880055177

After 350 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.260283854279

Change in best IC: 0 / Change in mean IC: -0.120005012242871

After 360 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.209688778613

Change in best IC: 0 / Change in mean IC: -0.0505950756654556

After 370 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.087855519222

Change in best IC: 0 / Change in mean IC: -0.121833259391281

After 380 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.034697986587
Change in best IC: 0 / Change in mean IC: -0.0531575326347991

After 390 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.034697986587

Change in best IC: 0 / Change in mean IC: 0

After 400 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 141.034122844699

Change in best IC: 0 / Change in mean IC: -0.00057514188802088

After 410 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.963006744586

Change in best IC: 0 / Change in mean IC: -0.0711161001134997

After 420 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.947972909578

Change in best IC: 0 / Change in mean IC: -0.0150338350081256

After 430 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.909318345826

Change in best IC: 0 / Change in mean IC: -0.0386545637518623

After 440 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.899154323515

Change in best IC: 0 / Change in mean IC: -0.0101640223109882

After 450 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.812209953658

Change in best IC: 0 / Change in mean IC: -0.0869443698566101

After 460 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.792425144782

Change in best IC: 0 / Change in mean IC: -0.0197848088761816

After 470 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.789625088012

Change in best IC: 0 / Change in mean IC: -0.00280005677046802

After 480 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.744701340208

Change in best IC: 0 / Change in mean IC: -0.0449237478030966

After 490 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.716693481452
Change in best IC: 0 / Change in mean IC: -0.0280078587563253

After 500 generations:
Best model: mpg~1+wt+qsec+gear+drat:hp+qsec:wt+gear:wt
Crit= 134.734029151824
Mean crit= 140.716693481452

Improvements in best and average IC have bebingo en below the specified goals.
Algorithm is declared to have converged.
Completed.
Code 
toc()
43.86 sec elapsed

Melhor Modelo

Cumpre recordar que, no início desta exposição, foi apresentada uma análise crítica da seleção sequencial, sugerindo-se que a metodologia disponível dentro do uso do pacote glmulti (força bruta e algoritmos genéticos) poderia ser superior. Assim, propõe-se agora uma comparação meticulosa entre os resultados obtidos pelos algoritmos exaustivo e genético e aqueles advindos das técnicas de seleção pra frente (forward) e pra trás (backward), com o intuito de determinar qual estratégia resulta no modelo de análise mais acurado e eficaz.

Code 
optimal_model_glmulti_exhaustive <- test_h@objects[[1]]
optimal_model_glmulti_genetic    <- test_g@objects[[1]]

compare_performance(
  optimal_model_glmulti_exhaustive, 
  optimal_model_glmulti_genetic, 
  optimal_model_backward, 
  optimal_model_forward
)
  • Melhor força-bruta:
Code 
optimal_model_glmulti_exhaustive$formula
mpg ~ 1 + wt + qsec + gear + drat:hp + qsec:wt + gear:wt
<environment: 0x000001bcb73baae8>
  • Melhor algo genético:
Code 
optimal_model_glmulti_genetic$formula
mpg ~ 1 + wt + qsec + gear + drat:hp + qsec:wt + gear:wt
<environment: 0x000001bcb1598080>
  • Melhor backward:
Code 
optimal_model_backward$formula
mpg ~ hp + drat + wt + qsec + gear + hp:drat + hp:wt + hp:qsec + 
    hp:gear + drat:wt + drat:qsec + wt:qsec + wt:gear
  • Melhor forward:
Code 
optimal_model_forward$formula
mpg ~ wt + hp + qsec + gear + wt:hp

Observa-se que a metodologia implementada pelo pacote glmulti resultou em valores inferiores tanto para o Critério de Informação de Akaike (AIC) quanto para o Critério de Informação Bayesiano (BIC), indicando uma eficácia notável. De forma interessante, o coeficiente de determinação (R²) obtido por glmulti situa-se medianamente entre os R²s advindos das seleções forward e backward, o que implica que os modelos gerados por glmulti não apresentam nem subajuste nem sobreajuste.

Em nosso estudo de caso, os algoritmos exaustivo e genético apresentaram resultados coincidentes (embora isso não seja uma constante) e identificaram três interações (drat:hp + qsec:wt + gear:wt) como significativas. Em contrapartida, a seleção backward indicou oito interações como relevantes, o que pode sugerir um sobreajuste, conforme refletido pelo R² elevado. Por outro lado, a seleção forward apontou apenas uma interação significativa, sugerindo um possível subajuste, em consonância com o R² mais baixo observado.

Assim sendo, almejo ter fornecido argumentos convincentes de que a abordagem glmulti supera a seleção Stepwise e culmina na elaboração de um modelo de análise verdadeiramente superior.

A saída de uma análise glmulti é um objeto que contém o set de modelos significativos (os 100 melhores modelos por padrão). Funções padrão de regressão do R como summary(), coef() ou plot() podem todas ser usadas para fazer uma inferência multi-modelo. Mas vamos começar com um breve resumo dos resultados que podem ser obtidos por meio do comando print():

Code 
print(h_model)
glmulti.analysis
Method: h / Fitting: glm / IC used: aicc
Level: 2 / Marginality: FALSE
From 100 models:
Best IC: 29793.4306133546
Best model:
[1] "wage ~ 1 + jobclass + education + health + health_ins + age + "  
[2] "    education:jobclass + health_ins:education + education:age + "
[3] "    health:age + health_ins:age"                                 
Evidence weight: 0.0786680339413555
Worst IC: 29801.3206286612
6 models within 2 IC units.
74 models to reach 95% of evidence weight.

… onde vemos as informações mais importantes, como a função de ajuste, os critérios de informação usados para classificar os modelos, a fórmula do melhor modelo e até o número de modelos que são tão bons quanto o melhor modelo. Existem 6 modelos, que também podemos ver se plotarmos nosso objeto:

Code 
plot(h_model)

Este gráfico mostra os valores dos Critérios de Informação (IC) para todos os 100 modelos do set de modelos significativos. Uma linha horizontal separa os 6 melhores modelos, que estão a menos de 2 unidades de IC de distância do MELHOR modelo. Mas quais preditores e interações esses 6 modelos contêm? Utilizando a função weightable, podemos exibi-los facilmente:

Code 
weightable(h_model)[1:6,] |>
  regulartable() |>       # mostra tabelas mais apresentáveis...
  autofit()

model

aicc

weights

wage ~ 1 + jobclass + education + health + health_ins + age + education:jobclass + health_ins:education + education:age + health:age + health_ins:age

29,793.43

0.07866803

wage ~ 1 + jobclass + education + health + health_ins + age + education:jobclass + health_ins:education + education:age + health:age

29,793.68

0.06952606

wage ~ 1 + jobclass + education + health_ins + age + education:jobclass + health_ins:education + education:age + health:age

29,794.40

0.04836431

wage ~ 1 + jobclass + education + health_ins + age + education:jobclass + health_ins:education + education:age + health:age + health_ins:age

29,794.43

0.04776916

wage ~ 1 + jobclass + education + health + health_ins + age + education:jobclass + health_ins:education + jobclass:age + education:age + health:age + health_ins:age

29,795.31

0.03070167

wage ~ 1 + jobclass + education + health + health_ins + age + education:jobclass + health_ins:jobclass + health_ins:education + education:age + health:age + health_ins:age

29,795.36

0.02990738

Neste momento, apresentamos as equações, os Critérios de Informação e os pesos de Akaike de nossos seis modelos mais destacados. O peso de Akaike de um modelo em particular indica a probabilidade de esse modelo ser o mais acertado entre todos os avaliados, servindo como um indicador de eficiência na minimização da perda de informação. Observa-se que, apesar de o modelo considerado “o melhor” possuir o maior peso, a diferença para o segundo colocado (e os subsequentes) não é significativa. Isso nos leva a questionar se o modelo mais bem classificado é de fato superior aos demais, dado que vários modelos apresentam plausibilidade similar. Surge, portanto, a questão: qual modelo deveríamos adotar?

Diante de seis modelos igualmente competentes, porém com distintas combinações de preditores e interações, identificar os elementos mais relevantes pode ser decisivo na escolha do modelo mais adequado. A função plot(), com o parâmetro type="s", facilita essa tarefa ao destacar a relevância dos termos do modelo de forma agregada. A relevância de um preditor ou interação é calculada pela soma dos pesos dos modelos que incluem tal variável. Portanto, uma variável frequente nos modelos de maior peso terá uma relevância elevada. Uma linha vertical em 80% é comumente utilizada para distinguir as variáveis de maior importância, embora esse ponto de corte seja arbitrário. Poderíamos, por exemplo, ajustar esse limite para 50% e considerar para o modelo final todos os preditores e interações com relevância superior a esse percentual.

Code 
plot(h_model, type = "s")

É digno de nota que o modelo inicial já incorpora a interação age:health_ins, atribuída com aproximadamente 50% de relevância. Esta escolha seria plenamente justificável. No entanto, considerando a presença de múltiplos termos com uma relevância próxima a 80%, opto por adotar apenas estes, incluindo a interação education:health_insurance e o preditor health, devido à sua distinta separação dos demais termos. Uma análise dos seis modelos mais eficazes revela que o segundo modelo contém precisamente esses termos. O terceiro modelo é ligeiramente inferior, pois omite a variável health, mas, dado que health é componente da interação mais significativa - age:health, sua inclusão é preferível. Portanto, em vez de aceitar de forma acrítica o modelo preeminente sugerido pelo algoritmo, procedemos a uma avaliação meticulosa dos resultados e decidimos, com base sólida, adotar o segundo modelo como O NOSSO MODELO IDEAL.

Estamos agora em posição de interpretar, visualizar e avaliar as premissas do NOSSO MODELO IDEAL com a diligência habitual.

Code 
best_model <- h_model@objects[[2]]

car::Anova(best_model)
Code 
plot_model(best_model, type = "int") |>
  plot_grid()

No R temos o pacote performance que facilita e muito a investigação das inferências de um modelo de regressão. Vale a pena conhecê-lo um pouco melhor.

Conclusão

Neste estudo, exploramos metodologias avançadas de seleção de modelos utilizando o pacote glmulti no ambiente estatístico R, com o objetivo de identificar o modelo mais adequado para explicar a variação nos salários de trabalhadores americanos. A abordagem de “força bruta”, que considera todas as combinações possíveis de preditores e suas interações, foi comparada com métodos de seleção passo a passo e algoritmos genéticos, revelando insights significativos sobre a adequação e eficácia dos modelos.

A análise exaustiva, embora computacionalmente mais intensiva, mostrou-se uma ferramenta valiosa, capaz de fornecer modelos com valores de Critério de Informação mais precisos, refletindo um equilíbrio entre a complexidade do modelo e a qualidade do ajuste. Por outro lado, o algoritmo genético emergiu como uma alternativa robusta, especialmente útil quando lidamos com um grande número de preditores numéricos, oferecendo resultados quase idênticos em uma fração do tempo.

Curiosamente, a análise revelou que a inclusão de termos altamente multicolineares, como o exemplo do peso e índice de massa corporal (IMC), poderia inflar desnecessariamente a complexidade do modelo sem adicionar valor interpretativo. Isso ressalta a importância de uma seleção criteriosa de preditores para evitar o sobreajuste e garantir a generalização do modelo.

Ao examinar os resultados, identificamos que o modelo com a interação age:health_ins (idade:seguro_saúde) e o preditor health (saúde), apesar de não ser o modelo com o menor Critério de Informação, oferece um equilíbrio entre a simplicidade e a capacidade de explicação, destacando-se como o modelo ideal para a nossa análise. Este modelo não apenas captura as nuances dos dados mas também evita a armadilha do overfitting, sugerindo que os termos selecionados representam relações genuínas e não artefatos dos dados.

A escolha cuidadosa do nosso modelo ideal, portanto, não foi o resultado de uma aceitação acrítica dos resultados algorítmicos, mas de uma consideração ponderada das evidências estatísticas e da relevância prática. A capacidade de interpretar, visualizar e verificar as premissas do modelo reforça a confiança em sua aplicabilidade e utilidade.

Em suma, este trabalho demonstra que a seleção de modelos, quando conduzida com rigor metodológico e uma compreensão profunda dos dados e das técnicas estatísticas, pode levar a descobertas significativas e modelos robustos, capazes de informar decisões práticas e teóricas no campo da economia laboral.

 

 

References

Siddiqi, Muhammad & Alsayat, Ahmed & Alhwaiti, Yousef & Azad, Mohammad & Alruwaili, Madallah & Alanazi, Saad & Kamruzzaman, MM & Khan, Asfandyar. (2022). A Precise Medical Imaging Approach for Brain MRI Image Classification. Computational Intelligence and Neuroscience. 2022. 1-15. 10.1155/2022/6447769. Disponível em https://www.researchgate.net

Gujarati, D., N. (2004) Basic Econometrics, fourth edition, The McGraw−Hill Companies

Hebbali A (2020). olsrr: Tools for Building OLS Regression Models. R package version 0.5.3, https://CRAN.R-project.org/package=olsrr.

Hyndman, R.J., & Athanasopoulos, G. (2021) Forecasting: principles and practice, 3rd edition, OTexts: Melbourne, Australia. OTexts.com/fpp3. Accessed on Nov 2023.

Minhas, M., S. Techniques for handling underfitting and overfitting in Machine Learning, Medium, Jun 5, 2021. Disponível em medium.com, Acesso em Out/2023.

Zablotski (2023, Nov. 5). yuzaR-Blog: R package reviews glmulti find the best model!. Retrieved from https://yuzar-blog.netlify.app/posts/2022-05-31-glmulti/

R packages

Code 
citation(package = "glmulti")

To cite package 'glmulti' in publications use:

  Calcagno V (2020). _glmulti: Model Selection and Multimodel Inference
  Made Easy_. R package version 1.0.8,
  <https://CRAN.R-project.org/package=glmulti>.

A BibTeX entry for LaTeX users is

  @Manual{,
    title = {glmulti: Model Selection and Multimodel Inference Made Easy},
    author = {Vincent Calcagno},
    year = {2020},
    note = {R package version 1.0.8},
    url = {https://CRAN.R-project.org/package=glmulti},
  }
Code 
# Total timing to compile this Quarto document

Sys.time() - start_time
Time difference of 3.946595 mins