Data Science Stack Exchange Community Digest

Top new questions this week:

Could you explain if this plot is good or bad. It is a sentiment analysis modelusing LSTM layers

Could you explain if this plot is good or bad. It is a sentiment analysis modelusing LSTM layers.20% of the data used for validation, and the remaining 80% for training.

machine-learning lstm matplotlib

asked by SDhakZZZ Score of 3 answered by Tripartio Score of 5

Problem with data representing classes that weren't present during supervised training

During prediction phase, fully trained supervised models may have to deal with data representing new classes, that weren’t part of the training and test sets. A real world example for this issue is …

machine-learning deep-learning supervised-learning

asked by user1934212 Score of 2 answered by Brian Spiering Score of 2

How to summarize a long text using GPT-3

What is the best way to summarize a long text that exceeds 4096 token limit (like a podcast transcript for example)? As I understand I need to split the text into chunks to summarize, and then …

openai-gpt automatic-summarization

asked by Poma Score of 1 answered by lynx Score of 0

Using the whole GloVe pre-trained embedding matrix or minimize the matrix based on the number of words in vocabulary

I have created a neural network for sentiment analysis using bidirectional LSTM layers and pre-trained GloVe embeddings. During the training I noticed that the …

deep-learning nlp lstm rnn word-embeddings

asked by NikSp Score of 1 answered by Brian Spiering Score of 1

What is the difference between the CCA weights and rotations?

I have been looking at the scikit learn Canonical Correlation Analysis (CCA) algorithm, and I have come across the terms “weights” and “rotations” as parameters of the CCA model. …

scikit-learn

asked by Felaix Score of 1 answered by Brian Spiering Score of 0

Should value of equation outcome be treated as variable for Random Forest model training?

For example, I got 5 variables, A to E, for prediction of a value. A B C = A – B D E the output of random forest rank C, B and A are variables with the most importance in descending order, my question …

random-forest

asked by Stephen C Score of 1 answered by Brian Spiering Score of 1

Dictionary-based text analysis- dealing with length

I am working on an analysis using a dictionary-based text-as-data approach. I have a dataset of texts (n=1200), and I am applying a dictionary of 50 words (I tokenize the text with each word being one …

regression normalization text dictionary

asked by Iamembarassed123 Score of 1 answered by Erwan Score of 0

Greatest hits from previous weeks:

How to set class weights for imbalanced classes in Keras?

I know that there is a possibility in Keras with the class_weights parameter dictionary at fitting, but I couldn’t find any example. Would somebody so kind to …

deep-learning classification keras weighted-data

asked by Hendrik Score of 251 answered by layser Score of 214

What is the Q function and what is the V function in reinforcement learning?

It seems to me that the $V$ function can be easily expressed by the $Q$ function and thus the $V$ function seems to be superfluous to me. However, I’m new to reinforcement learning so I guess I got …

machine-learning reinforcement-learning

asked by Martin Thoma Score of 65 answered by Juan Leni Score of 34

How to fill missing value based on other columns in Pandas dataframe?

Suppose I have a 5*3 data frame in which third column contains missing value 1 2 3 4 5 NaN 7 8 9 3 2 NaN 5 6 NaN I hope to generate value for missing value based …

pandas

asked by KyL Score of 29 answered by Icyblade Score of 32

What is the use of torch.no_grad in pytorch?

I am new to pytorch and started with this github code. I do not understand the comment in line 60-61 in the code …

python pytorch

asked by mausamsion Score of 66 answered by Adrien D Score of 55

When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?

machine-learning decision-trees information-theory

asked by Krish Mahajan Score of 107 answered by Dawny33 Score of 75

AttributeError: 'numpy.ndarray' object has no attribute 'columns'

…

scikit-learn pandas numpy

asked by Balu Score of 2 answered by Djib2011 Score of 3

SVM using scikit learn runs endlessly and never completes execution

I am trying to run SVR using scikit-learn (python) on a training dataset that has 595605 rows and 5 columns (features) while the test dataset has 397070 rows. The data has been pre-processed and …

python svm scikit-learn

asked by tejaskhot Score of 114 answered by Jessica Collins Score of 98

Can you answer these questions?

Can we add baseline to SHAP?

I have a doubt. I am currently using an integrated gradient for the DNN model for explainability. In that, we can specify the baseline as a parameter to the function. I am using all zeros for this. I …

shap explainable-ai

asked by Pritam Sinha Score of 1 answered by Jonas Mueller Score of 0

Does minimizing kl divergence (i.e. keep approximate posterior close to prior) contradict the goal of avoiding posterior collapse?

Posterior collapse means the variational distribution collapse towards the prior: $\exists i: s.t. \forall x: q_{\phi}(z_i|x) \approx p(z_i)$. $z$ becomes independent of $x$. We would like to avoid it …

vae

asked by JXuan Score of 1 answered by Dan Score of 0

How to actually begin using my first EC2 instance on AWS in order to run regressions too big for me to run locally

I am using the RStudio Server Amazon Machine Image (AMI) for a collaborative statistical learning research project intended to produce a paper for publication because its computational requirements …

regression feature-selection rstudio aws cloud-computing

asked by Marlen Score of 1