據(jù)集處理與鄰接矩陣生成：基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 (1))
交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南文章目錄交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 數(shù)據(jù)集結(jié)構(gòu)與文件說明文件目錄結(jié)構(gòu) 一、原始數(shù)據(jù)加載與預(yù)處理1. generate_training_data.py 示例METR-LA 二、滑動窗口劃分訓(xùn)練、驗證、測試集2. sliding_window_split.py 三、根據(jù)距離和連接關(guān)系生成鄰接矩陣3. adjacency_matrix.py 四、將 .npz 文件轉(zhuǎn)換為 .csv 文件以便查看4. npz_to_csv.py注意事項 六、參考交通速度數(shù)據(jù)集METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M含原始數(shù)據(jù)通過滑動窗口劃分train、val、test數(shù)據(jù)代碼根據(jù)距離和連接關(guān)系生成鄰接矩陣代碼。添加了npz_to_csv代碼目的是查看數(shù)據(jù) 交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 數(shù)據(jù)集結(jié)構(gòu)與文件說明文件目錄結(jié)構(gòu)traffic_speed_dataset/ ├── METR-LA/ │ ├── generate_training_data.py │ └── metr-la.h5 ├── PEMS-BAY/ │ ├── generate_training_data.py │ └── pems-bay.h5 └── utils/ ├── npz_to_csv.py ├── sliding_window_split.py └── adjacency_matrix.py 一、原始數(shù)據(jù)加載與預(yù)處理1.generate_training_data.py示例METR-LAimporth5pyimportnumpyasnpdefload_traffic_data(file_path):withh5py.File(file_path,r)asf:dataf[data][:]timestampsf[date][:]returndata,timestampsif__name____main__:file_pathmetr-la.h5data,timestampsload_traffic_data(file_path)print(fData shape:{data.shape})print(fTimestamps shape:{timestamps.shape}) 二、滑動窗口劃分訓(xùn)練、驗證、測試集2.sliding_window_split.pyimportnumpyasnpdefsliding_window_split(data,window_size,stride1,train_ratio0.6,val_ratio0.2):n_samples(len(data)-window_size)//stride1X[]y[]foriinrange(0,n_samples*stride,stride):X.append(data[i:iwindow_size])y.append(data[iwindow_size])Xnp.array(X)ynp.array(y)# 劃分訓(xùn)練、驗證、測試集total_sizelen(X)train_sizeint(total_size*train_ratio)val_sizeint(total_size*val_ratio)X_train,y_trainX[:train_size],y[:train_size]X_val,y_valX[train_size:train_sizeval_size],y[train_size:train_sizeval_size]X_test,y_testX[train_sizeval_size:],y[train_sizeval_size:]return(X_train,y_train),(X_val,y_val),(X_test,y_test)if__name____main__:datanp.random.rand(1000,10)# 示例數(shù)據(jù)實際應(yīng)替換為真實數(shù)據(jù)window_size12(X_train,y_train),(X_val,y_val),(X_test,y_test)sliding_window_split(data,window_size)print(fTrain set shape:{X_train.shape},{y_train.shape})print(fValidation set shape:{X_val.shape},{y_val.shape})print(fTest set shape:{X_test.shape},{y_test.shape}) 三、根據(jù)距離和連接關(guān)系生成鄰接矩陣3.adjacency_matrix.pyimportnumpyasnpimportscipy.sparseasspdefgenerate_adjacency_matrix(distance_file,sensor_ids,normalized_k0.1): Generate an adjacency matrix from a list of distances. :param distance_file: str, path to the txt file containing sensor distances. :param sensor_ids: list, list of sensor IDs. :param normalized_k: float, parameter for normalization. :return: np.ndarray, adjacency matrix. withopen(distance_file,r)asf:linesf.readlines()num_sensorslen(sensor_ids)dist_mxnp.zeros((num_sensors,num_sensors),dtypenp.float32)dist_mx[:]np.infforlineinlines:_,sensor_id,neighbor_sensor_id,distanceline.strip().split( )ifsensor_idnotinsensor_idsorneighbor_sensor_idnotinsensor_ids:continuedist_mx[sensor_ids.index(sensor_id)][sensor_ids.index(neighbor_sensor_id)]float(distance)dist_mx[sensor_ids.index(neighbor_sensor_id)][sensor_ids.index(sensor_id)]float(distance)distancesdist_mx[~np.isinf(dist_mx)].flatten()stddistances.std()adj_mxnp.exp(-np.square(dist_mx/std))# Make the adjacent matrix symmetric by taking the max.adj_mxnp.maximum.reduce([adj_mx,adj_mx.T])# Sets entries that lower than a threshold, i.e., k, to zero for sparsity.adj_mx[adj_mxnormalized_k]0returnadj_mxif__name____main__:distance_filedata/sensor_graph/adj_mx.txt# 替換為你的距離文件路徑sensor_ids[sensor_1,sensor_2,sensor_3]# 替換為你的傳感器ID列表adj_mxgenerate_adjacency_matrix(distance_file,sensor_ids)print(Adjacency Matrix:\n,adj_mx) 四、將.npz文件轉(zhuǎn)換為.csv文件以便查看4.npz_to_csv.pyimportnumpyasnpimportpandasaspddefnpz_to_csv(npz_file,csv_file):datanp.load(npz_file)dfpd.DataFrame(datadata[arr_0])df.to_csv(csv_file,indexFalse)if__name____main__:npz_filedata.npz# 替換為你的 .npz 文件路徑csv_filedata.csv# 輸出的 .csv 文件路徑npz_to_csv(npz_file,csv_file)print(fConverted{npz_file}to{csv_file})注意事項確保所有依賴項已安裝numpy,pandas,scipy,h5py。根據(jù)實際數(shù)據(jù)集調(diào)整參數(shù)如window_size,stride,normalized_k等。對于不同的數(shù)據(jù)集如 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M可能需要調(diào)整部分代碼以適應(yīng)其特定格式和特性。 六、參考METR-LA 數(shù)據(jù)集官方文檔PEMS-BAY 數(shù)據(jù)集官方文檔Scikit-learn 官方文檔NumPy 官方文檔Pandas 官方文檔 交通速度數(shù)據(jù)集處理與鄰接矩陣生成：基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 (1)
交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南文章目錄交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 數(shù)據(jù)集結(jié)構(gòu)與文件說明文件目錄結(jié)構(gòu) 一、原始數(shù)據(jù)加載與預(yù)處理1. generate_training_data.py 示例METR-LA 二、滑動窗口劃分訓(xùn)練、驗證、測試集2. sliding_window_split.py 三、根據(jù)距離和連接關(guān)系生成鄰接矩陣3. adjacency_matrix.py 四、將 .npz 文件轉(zhuǎn)換為 .csv 文件以便查看4. npz_to_csv.py注意事項 六、參考交通速度數(shù)據(jù)集METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M含原始數(shù)據(jù)通過滑動窗口劃分train、val、test數(shù)據(jù)代碼根據(jù)距離和連接關(guān)系生成鄰接矩陣代碼。添加了npz_to_csv代碼目的是查看數(shù)據(jù) 交通速度數(shù)據(jù)集處理與鄰接矩陣生成基于 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M 的完整代碼指南 數(shù)據(jù)集結(jié)構(gòu)與文件說明文件目錄結(jié)構(gòu)traffic_speed_dataset/ ├── METR-LA/ │ ├── generate_training_data.py │ └── metr-la.h5 ├── PEMS-BAY/ │ ├── generate_training_data.py │ └── pems-bay.h5 └── utils/ ├── npz_to_csv.py ├── sliding_window_split.py └── adjacency_matrix.py 一、原始數(shù)據(jù)加載與預(yù)處理1.generate_training_data.py示例METR-LAimporth5pyimportnumpyasnpdefload_traffic_data(file_path):withh5py.File(file_path,r)asf:dataf[data][:]timestampsf[date][:]returndata,timestampsif__name____main__:file_pathmetr-la.h5data,timestampsload_traffic_data(file_path)print(fData shape:{data.shape})print(fTimestamps shape:{timestamps.shape}) 二、滑動窗口劃分訓(xùn)練、驗證、測試集2.sliding_window_split.pyimportnumpyasnpdefsliding_window_split(data,window_size,stride1,train_ratio0.6,val_ratio0.2):n_samples(len(data)-window_size)//stride1X[]y[]foriinrange(0,n_samples*stride,stride):X.append(data[i:iwindow_size])y.append(data[iwindow_size])Xnp.array(X)ynp.array(y)# 劃分訓(xùn)練、驗證、測試集total_sizelen(X)train_sizeint(total_size*train_ratio)val_sizeint(total_size*val_ratio)X_train,y_trainX[:train_size],y[:train_size]X_val,y_valX[train_size:train_sizeval_size],y[train_size:train_sizeval_size]X_test,y_testX[train_sizeval_size:],y[train_sizeval_size:]return(X_train,y_train),(X_val,y_val),(X_test,y_test)if__name____main__:datanp.random.rand(1000,10)# 示例數(shù)據(jù)實際應(yīng)替換為真實數(shù)據(jù)window_size12(X_train,y_train),(X_val,y_val),(X_test,y_test)sliding_window_split(data,window_size)print(fTrain set shape:{X_train.shape},{y_train.shape})print(fValidation set shape:{X_val.shape},{y_val.shape})print(fTest set shape:{X_test.shape},{y_test.shape}) 三、根據(jù)距離和連接關(guān)系生成鄰接矩陣3.adjacency_matrix.pyimportnumpyasnpimportscipy.sparseasspdefgenerate_adjacency_matrix(distance_file,sensor_ids,normalized_k0.1): Generate an adjacency matrix from a list of distances. :param distance_file: str, path to the txt file containing sensor distances. :param sensor_ids: list, list of sensor IDs. :param normalized_k: float, parameter for normalization. :return: np.ndarray, adjacency matrix. withopen(distance_file,r)asf:linesf.readlines()num_sensorslen(sensor_ids)dist_mxnp.zeros((num_sensors,num_sensors),dtypenp.float32)dist_mx[:]np.infforlineinlines:_,sensor_id,neighbor_sensor_id,distanceline.strip().split( )ifsensor_idnotinsensor_idsorneighbor_sensor_idnotinsensor_ids:continuedist_mx[sensor_ids.index(sensor_id)][sensor_ids.index(neighbor_sensor_id)]float(distance)dist_mx[sensor_ids.index(neighbor_sensor_id)][sensor_ids.index(sensor_id)]float(distance)distancesdist_mx[~np.isinf(dist_mx)].flatten()stddistances.std()adj_mxnp.exp(-np.square(dist_mx/std))# Make the adjacent matrix symmetric by taking the max.adj_mxnp.maximum.reduce([adj_mx,adj_mx.T])# Sets entries that lower than a threshold, i.e., k, to zero for sparsity.adj_mx[adj_mxnormalized_k]0returnadj_mxif__name____main__:distance_filedata/sensor_graph/adj_mx.txt# 替換為你的距離文件路徑sensor_ids[sensor_1,sensor_2,sensor_3]# 替換為你的傳感器ID列表adj_mxgenerate_adjacency_matrix(distance_file,sensor_ids)print(Adjacency Matrix:\n,adj_mx) 四、將.npz文件轉(zhuǎn)換為.csv文件以便查看4.npz_to_csv.pyimportnumpyasnpimportpandasaspddefnpz_to_csv(npz_file,csv_file):datanp.load(npz_file)dfpd.DataFrame(datadata[arr_0])df.to_csv(csv_file,indexFalse)if__name____main__:npz_filedata.npz# 替換為你的 .npz 文件路徑csv_filedata.csv# 輸出的 .csv 文件路徑npz_to_csv(npz_file,csv_file)print(fConverted{npz_file}to{csv_file})注意事項確保所有依賴項已安裝numpy,pandas,scipy,h5py。根據(jù)實際數(shù)據(jù)集調(diào)整參數(shù)如window_size,stride,normalized_k等。對于不同的數(shù)據(jù)集如 METR-LA、PEMS-BAY、PEMSD7L、PEMSD7M可能需要調(diào)整部分代碼以適應(yīng)其特定格式和特性。 六、參考METR-LA 數(shù)據(jù)集官方文檔PEMS-BAY 數(shù)據(jù)集官方文檔Scikit-learn 官方文檔NumPy 官方文檔Pandas 官方文檔