Data Science Stack Exchange Community Digest

Top new questions this week:

Resources for Promotion/Demotion Strategies for ML Item Recommendation Systems?

We are looking to design a system where specific items or categories of items can be boosted/promoted up or relegated/demoted down the recommendation order. What are the common strategies or standards …

machine-learning data-science-model recommender-system reference-request research

asked by JPTheEngineer Score of 4 answered by Brian Spiering Score of 0

What is the purpose of EarlyStopping returning last epoch's weights by default?

I recently realized that keras callback for early stopping returns the last epoch’s weights by default. If you want to do otherwise you can use the argument …

machine-learning deep-learning keras early-stopping

asked by Luca Clissa Score of 3 answered by noe Score of 7

Merging data approach in Data Science projects

This is more of an infrastructural question about data science. How would you manage data merging in your GitHub repository? As an example, as a data scientist I might be working on my branch and …

dataset version-control management git

asked by Mattia Surricchio Score of 2 answered by 0xedu Score of 0

problem with standardScaler

problem with standardScaler hi I’d like to scale one column in the titanic data set. I am using the following code segment. for some reason df_scaled results an empty set. how can I solve it? what is …

scikit-learn feature-scaling

asked by Mehmet Deniz Score of 1 answered by Quan Nguyen Score of 1

Customer Segmentation with mixed data

I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), …

machine-learning clustering categorical-data feature-scaling

asked by ArgyGr Score of 1 answered by 0xedu Score of 0

scikit learn logistic regression to annotate single cell RNA seq data

I want to use scikit learn logistic regression to train a model on a labelled single cell RNA sample and subsequently apply this model on new unlabelled single cell RNA seq samples to annotate the …

python scikit-learn logistic-regression

asked by LabDancer Score of 1

Rule based vs predictive maintenance models

I have data for pumps which have one or more sensors to record the air pressure. Apart from the sensor_id and timestamp, with …

machine-learning predictive-modeling machine-learning-model

asked by SageMaker Score of 1 answered by Nemo_the_scientist Score of 0

Greatest hits from previous weeks:

When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?

machine-learning decision-trees information-theory

asked by Krish Mahajan Score of 107 answered by Dawny33 Score of 75

Merging multiple data frames row-wise in PySpark

I have 10 data frames pyspark.sql.dataframe.DataFrame, obtained from randomSplit as …

python apache-spark cross-validation pyspark

asked by krishna Prasad Score of 34 answered by Jan van der Vegt Score of 57

What is the difference between Gradient Descent and Stochastic Gradient Descent?

What is the difference between Gradient Descent and Stochastic Gradient Descent? I am not very familiar with these, can you describe the difference with a short example?

machine-learning neural-network deep-learning gradient-descent

asked by Developer Score of 75 answered by Sociopath Score of 83

What is a channel in a CNN?

I was reading an article about convolutional neural networks, and I found something that I don’t understand, which is: The filter must have the same number of channels as the input image so that the …

machine-learning neural-network cnn

asked by J.D. Score of 20 answered by noe Score of 23

How to draw Deep learning network architecture diagrams?

I have built my model. Now I want to draw the network architecture diagram for my research paper. Example is shown below:

machine-learning neural-network deep-learning svm software-recommendation

asked by Muhammad Ali Score of 187 answered by Pablo Rivas Score of 133

Train Accuracy vs Test Accuracy vs Confusion matrix

After I developed my predictive model using Random Forest I get the following metrics: …

python predictive-modeling accuracy confusion-matrix classifier

asked by Pedro Alves Score of 16 answered by S van Balen Score of 25

How to plot two columns of single DataFrame on Y axis

I have two data frames (Action, Comedy). Action contains two columns (year, rating) ratings columns contains average rating with respect to year. The Comedy data frame contains the same two columns …

python pandas visualization dataframe

asked by Bilal Butt Score of 18 answered by user2333453 Score of 20

Can you answer these questions?

Underfitting and perfomance metrics in unsupervised methods

My question is simple and yet quite hard to find an answer to. In an unsupervised method, for example, when you have to reconstruct an input, how can you tell if your loss is good enough? Generally, …

deep-learning unsupervised-learning anomaly-detection overfitting hyperparameter-tuning

asked by BilboBuggins Score of 1

Can the Apple M1's iGPU access the entire RAM as “video memory” when training with typical deep learning frameworks?

Can the Apple M1’s iGPU access the entire RAM as “video memory” when training with typical deep learning frameworks (e.g., tensorflow_macos)? If not, what memory do they use as video memory?

training gpu memory

asked by Franck Dernoncourt Score of 1

I have 2 Columns of text, Should I use different vectorizer and Embeddings for each or just one?

I have a dataset with two input columns as text. Should I use same textvectorizer in both columns or different ones? I am asking this because. columns a has average …

deep-learning nlp tensorflow text-generation

asked by Tikendra Kumar Sahu Score of 1 answered by Brian Spiering Score of 0