["python","pandas","dataframe"]
Pandas convert a column of list to dummies

I have a dataframe where one column is a list of groups each of my users belongs to. Something like: index groups 0 ['a','b','c'] 1 ['c'] 2 ['b','c','e'] 3 ['a','c'] 4 ['b','e'] And what I would like to do is create a series of dummy columns to identify which groups each user belongs to in order to run some analyses index a b c d e 0 1 1 1 0 0 1 0 0 1 0 0 2 0 1 1 0 1 3 1 0 1 0 0 4 0 1 0 0 0 pd.get_dummies(df['groups']) won't work because that just returns a column for each different list in my column. The solution needs to be efficient as the dataframe will contain 500,000+ rows.
Using s for your df['groups']: In [21]: s = pd.Series({0: ['a', 'b', 'c'], 1:['c'], 2: ['b', 'c', 'e'], 3: ['a', 'c'], 4: ['b', 'e'] }) In [22]: s Out[22]: 0 [a, b, c] 1 [c] 2 [b, c, e] 3 [a, c] 4 [b, e] dtype: object This is a possible solution: In [23]: pd.get_dummies(s.explode()).groupby(level=0).sum() Out[23]: a b c e 0 1 1 1 0 1 0 0 1 0 2 0 1 1 1 3 1 0 1 0 4 0 1 0 1 The logic of this is: .explode() flattens the series of lists to a series of single values (with the index keeping track of the original row number) pd.get_dummies( ) creating the dummies .groupby(level=0).sum() for combining the different rows that should be one row (by summing up grouped by the index (level=0), i.e. the original row number)) If this will be efficient enough, I don't know, but in any case, if performance is important, storing lists in a dataframe is not a very good idea. Updates since original answer Since version 0.25, s.explode() can be used to flatten the Series of lists, instead of the original s.apply(pd.Series).stack() Since version 1.3.0, using the level keyword in aggregations is deprecated and will be removed from newer versions soon, so is recommended to use df.groupby(level=0).sum() instead of df.sum(level=0)