["python","pandas","datetime","time-series"]
Combine Date and Time columns using pandas

I have a pandas dataframe with the following columns: data = {'Date': ['01-06-2013', '02-06-2013', '02-06-2013', '02-06-2013', '02-06-2013', '03-06-2013', '03-06-2013', '03-06-2013', '03-06-2013', '04-06-2013'], 'Time': ['23:00:00', '01:00:00', '21:00:00', '22:00:00', '23:00:00', '01:00:00', '21:00:00', '22:00:00', '23:00:00', '01:00:00']} df = pd.DataFrame(data) Date Time 0 01-06-2013 23:00:00 1 02-06-2013 01:00:00 2 02-06-2013 21:00:00 3 02-06-2013 22:00:00 4 02-06-2013 23:00:00 5 03-06-2013 01:00:00 6 03-06-2013 21:00:00 7 03-06-2013 22:00:00 8 03-06-2013 23:00:00 9 04-06-2013 01:00:00 How do I combine data['Date'] & data['Time'] to get the following? Is there a way of doing it using pd.to_datetime? Date 01-06-2013 23:00:00 02-06-2013 01:00:00 02-06-2013 21:00:00 02-06-2013 22:00:00 02-06-2013 23:00:00 03-06-2013 01:00:00 03-06-2013 21:00:00 03-06-2013 22:00:00 03-06-2013 23:00:00 04-06-2013 01:00:00
It's worth mentioning that you may have been able to read this in directly e.g. if you were using read_csv using parse_dates=[['Date', 'Time']]. Assuming these are just strings you could simply add them together (with a space), allowing you to use to_datetime, which works without specifying the format= parameter In [11]: df['Date'] + ' ' + df['Time'] Out[11]: 0 01-06-2013 23:00:00 1 02-06-2013 01:00:00 2 02-06-2013 21:00:00 3 02-06-2013 22:00:00 4 02-06-2013 23:00:00 5 03-06-2013 01:00:00 6 03-06-2013 21:00:00 7 03-06-2013 22:00:00 8 03-06-2013 23:00:00 9 04-06-2013 01:00:00 dtype: object In [12]: pd.to_datetime(df['Date'] + ' ' + df['Time']) Out[12]: 0 2013-01-06 23:00:00 1 2013-02-06 01:00:00 2 2013-02-06 21:00:00 3 2013-02-06 22:00:00 4 2013-02-06 23:00:00 5 2013-03-06 01:00:00 6 2013-03-06 21:00:00 7 2013-03-06 22:00:00 8 2013-03-06 23:00:00 9 2013-04-06 01:00:00 dtype: datetime64[ns] Alternatively, without the + ' ', but the format= parameter must be used. Additionally, pandas is good at inferring the format to be converted to a datetime, however, specifying the exact format is faster. pd.to_datetime(df['Date'] + df['Time'], format='%m-%d-%Y%H:%M:%S') Note: surprisingly (for me), this works fine with NaNs being converted to NaT, but it is worth worrying that the conversion (perhaps using the raise argument). %%timeit # sample dataframe with 10000000 rows using df from the OP df = pd.concat([df for _ in range(1000000)]).reset_index(drop=True) %%timeit pd.to_datetime(df['Date'] + ' ' + df['Time']) [result]: 1.73 s ± 10.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) %%timeit pd.to_datetime(df['Date'] + df['Time'], format='%m-%d-%Y%H:%M:%S') [result]: 1.33 s ± 9.88 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)