Data Science Stack Exchange Community Digest

Top new questions this week:

Does fine-tuning require retraining the entire model?

Would it be necessary to retrain the entire model if we were to perform fine-tuning? Let’s say we somehow got the GPT-3 model from OpenAI (I know GPT-3 is closed source). Would anyone with access to a …

machine-learning deep-learning transformer openai-gpt

asked by Exploring Score of 4 answered by Andy Score of 9

Features for POS tagging

I am building an NLP model which uses MEMM in order to tag parts of speech. My model uses history of two previous words and tags, and of the next word, alongside the current word and tag. I used those …

nlp

asked by Daniel Score of 2 answered by Brian Spiering Score of 2

Keras model does not construct the layers in sequence

Every time I print the model summary it prints out the convlstm first and then batchnormalization at the end. …

deep-learning keras tensorflow

asked by pwnkit Score of 2 answered by Lynn Score of 4

Why am I getting the exact same results with both a Logistic Regression and Decision Tree Classifier?

I am working on a binary classification problem and am using sklearn’s logistic regression model and decision tree classifier. Somehow I am getting the exact same results and accuracy score on both. I …

scikit-learn decision-trees logistic-regression

asked by RandomGuy Score of 2 answered by liakoyras Score of 3

Neural Network Hidden Layers an data Dimensionality

In multi-Linear regression where we have a model of $y=X_1+X_2$ it’s a common practice (as I studied in my Master’s) to increase the dimensionality and try to use a model of $y=X_1+X_2+X_1 X_2$. Can …

deep-learning neural-network statistics

asked by mrcoet Score of 1 answered by Lynn Score of 0

why it would be improper to compute and use test set means?

I have 2 questions regarding the whole subject of the data set in machine learning and I would be happy to receive an answer 🙂 1.Why it would be improper to compute and use test set means ? 2.Why we …

machine-learning dataset training validation

asked by Yovel Score of 1 answered by Brian Spiering Score of 0

ROC Curve for model validation

Is there a general approach that the ROC curve can be used for to validate a model? My understanding is that we can use it to compare different threshold values to determine the best, or even see how …

logistic-regression binary-classification roc

asked by user143064 Score of 1 answered by Dave Score of 2

Greatest hits from previous weeks:

Merging multiple data frames row-wise in PySpark

I have 10 data frames pyspark.sql.dataframe.DataFrame, obtained from randomSplit as …

python apache-spark cross-validation pyspark

asked by krishna Prasad Score of 34 answered by Jan van der Vegt Score of 57

Perform k-means clustering over multiple columns

I am trying to perform k-means clustering on multiple columns. My data set is composed of 4 numerical columns and 1 categorical column. I already researched previous questions but the answers are not …

python clustering pandas

asked by Lola Score of 13 answered by Simon Larsson Score of 19

Number of parameters in an LSTM model

How many parameters does a single stacked LSTM have? The number of parameters imposes a lower bound on the number of training examples required and also influences the training time. Hence knowing the …

deep-learning rnn

asked by wabbit Score of 59

Opening a 20GB file for analysis with pandas

I am currently trying to open a file with pandas and python for machine learning purposes it would be ideal for me to have them all in a DataFrame. Now The file is 18GB large and my RAM is 32 GB but I …

python bigdata pandas anaconda

asked by Hari Prasad Score of 47 answered by Olel Daniel Score of 40

How do I compare columns in different data frames?

I would like to compare one column of a df with other df’s. The columns are names and last names. I’d like to check if a person in one data frame is in another one.

pandas dataframe

asked by a_a_a Score of 49 answered by Tarek Score of 38

What is the difference between Gradient Descent and Stochastic Gradient Descent?

What is the difference between Gradient Descent and Stochastic Gradient Descent? I am not very familiar with these, can you describe the difference with a short example?

machine-learning neural-network deep-learning gradient-descent

asked by Developer Score of 75 answered by Sociopath Score of 82

Remove part of string in R

I have a table in R. It just has two columns and many rows. Each element is a string that contains some characters and some numbers. I need number part of the element. How can I have number part? For …

r dataset bioinformatics

asked by mahtab Score of 3 answered by Marmite Bomber Score of 7

Can you answer these questions?

Inverse Scaling Partitioned Data

I have scaled an original matrix A with sklearn‘s StandardScaler, resulting to a matrix S. I then partitioned the result into …

python scikit-learn feature-scaling

asked by ccccc Score of 1 answered by liakoyras Score of 0

Which is the final model from Nested Cross Validation: Accuracy or Frequency?

www.cnblogs.com/guo-xiang/p/8044624.html explains with a nice example the mechanics of Nested Cross Validation. In the picture, the example shows how to use Nested CV for hyperparameter …

cross-validation model-selection

asked by Sm1 Score of 1 answered by Brian Spiering Score of 0