logo
down
shadow

Pandas+Python - How to know when a value changes?


Pandas+Python - How to know when a value changes?

By : NATE WANG
Date : November 20 2020, 11:01 PM
it fixes the issue I don't know if your entire data set is built the same way as the one you are showing us but from what I can see you are searching for occurrence of 3 to 1 in the m columns which would result in a difference of -2 :
code :
df[df['M'].diff()==-2].index
Out[101]: Int64Index([6, 13], dtype='int64')
df[df['M'].diff()<0].index
Out[103]: Int64Index([6, 13], dtype='int64')
df[(df['M'].diff()!=0) & (df['M']==1)].index
Out[104]: Int64Index([0, 6, 13], dtype='int64')


Share : facebook icon twitter icon
pandas python inserting part of a column to a column based on conditions pandas python

pandas python inserting part of a column to a column based on conditions pandas python


By : Mahalakshmi
Date : March 29 2020, 07:55 AM
help you fix your problem i have a large dataset that i want to work with , but here i am using a mock dataset: , Here is the code:
code :
# create the mock dataframe with 3 blocks

data1 = DataFrame({'Block': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    'Concentration': [100, 100, 100, 33, 33, 33, 100, 100, 100, 33, 33, 33, 0,0,0],
    'Name' : ['A', 'A',  'A', 'A', 'A', 'A', 'B', 'B',  'B', 'B', 'B', 'B', 'PB', 'PB', 'PB'],
    'value': [86, 194, 452, 140, 285, 2011, 8, 19, 45, 14, 28, 201, 100, 111, 222 ]})


data2 = data1.copy(); data2.Block = 2
data3 = data1.copy(); data3.Block = 3

data = pd.concat([data1, data2, data3], axis=0)

def temp1(df):
    df_others = df[df.Name != 'PB']
    df_pb = df[df.Name == 'PB']
    def temp2(dfx):
        df_app = df_pb.copy()
        df_app = df_app[df_app.Concentration==0] # in case name 'PB' have more than one concentrations
        df_app['Name'] = dfx['Name'].values[0] ## modified code
        df_pername = pd.concat([dfx, df_app])
        return df_pername
    df1 = df_others.groupby('Name', group_keys=False).apply(temp2)
    df2 = pd.concat([df1, df_pb])
    return df2

data_changed = data.groupby('Block', group_keys=False).apply(temp1)

data_changed.index = range(len(data_changed))

In [151]: data_changed
Out[151]: 
    Block  Concentration Name  value
0       1            100    A     86
1       1            100    A    194
2       1            100    A    452
3       1             33    A    140
4       1             33    A    285
5       1             33    A   2011
6       1              0    A    100
7       1              0    A    111
8       1              0    A    222
9       1            100    B      8
10      1            100    B     19
11      1            100    B     45
12      1             33    B     14
13      1             33    B     28
14      1             33    B    201
15      1              0    B    100
16      1              0    B    111
17      1              0    B    222
18      1              0   PB    100
19      1              0   PB    111
20      1              0   PB    222
..    ...            ...  ...    ...
58      3              0    B    111
59      3              0    B    222
60      3              0   PB    100
61      3              0   PB    111
62      3              0   PB    222

[63 rows x 4 columns]
What is the most efficient way to subtract rows of a dataframe (python pandas) by index in python pandas

What is the most efficient way to subtract rows of a dataframe (python pandas) by index in python pandas


By : Shishir
Date : March 29 2020, 07:55 AM
hop of those help? use diff with fillna:
code :
In [4]:
df['diff time'] = df['arrivalTime'].diff().fillna(0)
df

Out[4]:
   cEventID         arrivalTime  diff time
0   1167533                 NaT   00:00:00
1   1167541 2015-07-14 04:01:21   00:00:00
2   1167545 2015-07-14 04:03:20   00:01:59
3   1167549 2015-07-14 04:07:45   00:04:25
4   1167552 2015-07-14 04:10:21   00:02:36
5   1167553 2015-07-14 04:13:39   00:03:18
6   1167558 2015-07-14 04:15:58   00:02:19
7   1167561 2015-07-14 04:20:23   00:04:25
python-pandas: dealing with NaT type values in a date columns of pandas dataframe

python-pandas: dealing with NaT type values in a date columns of pandas dataframe


By : K. E.
Date : March 29 2020, 07:55 AM
it helps some times I have a dataframe with mixed datatype column, and I applied pd.to_datetime(df['DATE'],coerce=True) and got the below dataframe , Say you start with something like this:
code :
df = pd.DataFrame({
    'CUSTOMER_name': ['abc', 'def', 'abc', 'def', 'abc', 'fff'], 
    'DATE': ['NaT', 'NaT', '2010-04-15 19:09:08', '2011-01-25 15:29:37', '2010-04-10 12:29:02', 'NaT']})
df.DATE = pd.to_datetime(df.DATE)
>>> pd.to_datetime(df.DATE.groupby(df.CUSTOMER_name).min())
CUSTOMER_name
abc   2010-04-10 12:29:02
def   2011-01-25 15:29:37
fff                   NaT
Name: DATE, dtype: datetime64[ns]
>>> pd.to_datetime(df.DATE.groupby(df.CUSTOMER_name).min()).dt.date
Out[19]: 
CUSTOMER_name
abc    2010-04-10
def    2011-01-25
fff           NaN
Name: DATE, dtype: object
Python Pandas read_excel different behavior in parsing MultiIndex dataframe between Pandas 0.18.1 and Pandas > 0.19

Python Pandas read_excel different behavior in parsing MultiIndex dataframe between Pandas 0.18.1 and Pandas > 0.19


By : Nyago alex
Date : March 29 2020, 07:55 AM
I hope this helps . I am totally confused. Probably I miss an update in pandas API. , You could make your index semi-manually
Get the header
code :
header = pd.read_excel(fname, 'Sheet1', index=[0], header=None).iloc[:2, 1:].ffill(axis=1)
df = pd.read_excel(fname, 'Sheet1', skiprows=[0,1], index=0, header=None).rename(columns={0: 'A'}).set_index('A')
df.columns=pd.MultiIndex.from_arrays(header.values)
    B       D       F
    C   C   E   E   G   G
A                       
A1  X   Y   Z   U   J   K
A2  XX  YY  ZZ  UU  JJ  KK
A3  XXX     YYY     ZZZ     UUU     JJJ     KKK
Visual Studio Code windows , Python Pandas . No module named pandas

Visual Studio Code windows , Python Pandas . No module named pandas


By : Julia L
Date : March 29 2020, 07:55 AM
help you fix your problem It seems that the module pandas is installed in a virtual envorinment which you are not accessing via VS Code.
I'd suggest you to install pandas in default python as well via
code :
pip install pandas
Related Posts Related Posts :
  • Django Form Based on Variable Attributes
  • Relocate all the evens
  • How to scrap span ids' texts in beautifulsoup in the following html?
  • How to generate random number in a given range as a Tensorflow variable
  • Gradient Descent Variation doesn't work
  • Python 2.7 - search for a particular URL on a webpage with ajax
  • How to configure Luigi task retry correctly?
  • web.py : an urlencoded slash into args
  • Use of pyzmq's logging handler in python
  • How to count the number of a particular entry. python
  • devide int into lower whole ints
  • Access atribute of every object in pandas dataframe column
  • Combine Dataframe rows on conditions
  • Select closest date (or value) in pandas / python
  • Pycharm and remote interpreter (Docker) shows errors but runs fine
  • Get started to launch google-cloud-ml with my own dataset
  • Multiprocessing: use only the physical cores?
  • Django Login Custom Auth works locally but not on production server
  • Python: Invalid HTTP basic authentication header with long base64 string
  • How can I request several pages without wating for the output?
  • Flask Response vs Flask make_response
  • python linear regression predict by date
  • How to get pandas dataframe where columns are the subsequent n-elements from another column dataframe?
  • MYSQL: "Access denied for user 'X'@'localhost' (using password: YES)" PYTHON
  • install scipy package via pycharm in windows 10 64 bit - python 3.5
  • Update time in linux and solaris machines from robot framework
  • Complex pandas isin function
  • Averaging over every n elements of an array without numpy
  • An elegant way of inserting multiple arguments
  • IntegrityError:NOT NULL constraint failed: chatapp_chat.message
  • Indexing of 3d numpy arrays with 2d arrays
  • Creating a mean of columns with csv writer
  • Reading in environment variables from an environment file
  • Collapse duplicate rows with pandas
  • How can I use skyfied to convert SGP4 TEME coordinate to ECEF?
  • How to modify object in Python's Rtree index
  • Create Hexbin plot with pandas dataframe using index and columns names as x and y
  • SQLAlchemy query returns no data if a database field is empty
  • Python pandas column asignment between dataframe and series does not work
  • ValueError: Unknown label type: array while using Decision Tree Classifier and using a custom dataset
  • Trouble accessing exif information with PIL.Image._getexif()
  • Use all coordinates in a grid except with certain value
  • Why for loop is splitting strings of user input?
  • How can I add two variable and assign to result variable in Python?
  • Error when parsing timestamp with pandas read_csv
  • Slicing arrays based on boolean array in python
  • Feeding scipy.sparse() sparse matrices into CVXOPT
  • How to separate a irregularly cased string to get the words? - Python
  • Pandas: replace some values in column if that contain a substring
  • Fabric does not close the ssh connection
  • Python Creating Classes Code
  • When will train() method in easy_seq2seq stop?
  • How to split each element of the RDD in spark with python?
  • Read in csv file in python, round the values and write back to file
  • How to properly close a QWidget-window in an API with PythonQt
  • How to know which segment a value reside in
  • pandas: convert multiple categories to dummies
  • 'Options' object has no attribute 'get_all_field_names'
  • Customize django filter model field
  • NLTK tag Dutch sentence
  • shadow
    Privacy Policy - Terms - Contact Us © soohba.com