How to read a and create dataset from JSON files

1

I have many JSON files from which I have to create data sets, I can’t figure out how to handle this type of data in pythorch ?
Thank you.

2

It’d better to explain your json structure at least briefly.

1 Like
3

I have a list of files for every label, stored in a directory with the label name and every file contains this:

{“data”: [{
“ID”:,
“SensorName”: ,
“DigitalValue”: ,
“AnalogValue”: ,
“SchematicName”:,
“Pin”:,
“Enable”:,
“TimeStamp”:,
…
}]}

4
  1. Choose data you want to use. For example, ID, DigitalValue, …
  2. Convert them into number(integer, float, …)
  3. Make a list of them
  4. Convert that list into PyTorch Tensor
data = []
for k, v in x['data'][0].items():
    if type(v) == str:
        # convert it into int or float
        # and append into data
    else:
        data.append(v)

torch_data = torch.Tensor(data)
1 Like
5

thanks but, what about the labeling, how should I set the data to be sorted with its labels?

6

Labels follow the same but

list_of_labels = [0, 1, 2, 0, 1, 3, ..., 4]
torch_labels = torch.LongTensor(list_of_labels) # for classifier
torch_labels = torch.FloatTensor(list_of_labels) # for regression

Wish my comment will help but the question is too broad.
The keypoint is converting a list into tensor.

1 Like
7

Sorry for my too broad question, what I wanted to say is that every file has the structure I pasted in the previous answear, and for every label I have a list of file names so I was wondering how to read the data (you already answeared this thanks ), and add the label to this dataset ?