Как получить полные результаты с помощью API ClinicTrials.gov на Python?

Вы показываете код для old API с полями для new API и добавляете ссылку на документацию для new API, не показывая ссылку на документацию для old API (https://classic.clinicaltrials.gov/api/gui/ref/api_urls), так что у меня возникла путаница в моей голове.

Но наконец-то я нашел некоторую информацию.

В old API вам нужно изменить 'min_rnk' и 'max_rnk', чтобы получить следующие страницы.

Версия для old API:

Я использую цикл for, чтобы получить несколько страниц, но вам может понадобиться while True, чтобы получить все страницы.

Я также использую [:10], чтобы отображать только несколько результатов для каждой страницы — чтобы я мог легко увидеть, различаются ли они, — и я не получаю одну и ту же страницу много раз. Но, возможно, вам придется удалить его.

Я использую fields из примера в документации, потому что old API использует разные имена, и я не смог найти имена, которые вы используете в своем коде.

И я использую get(url, params) вместо создания строки с URL и всеми параметрами. И я использую response.json(), чтобы получить результат.

Я не загружаю его в pandas, потому что json имеет сложную структуру и для переформатирования его в таблицу потребуется дополнительный код. Может быть, мне следует получить это как CSV вместо JSON

import requests

base = 'https://clinicaltrials.gov/api/query/study_fields'   # old API

# old API fields
extract_fields = [
    'NCTId',
    'Condition',
    'BriefTitle',
]

# old API parameter
params = {
    'fields': ",".join(extract_fields), 
    'expr': 'Diabetes',
    'min_rnk': 1,
    'max_rnk': 1000,
    'fmt': 'json', 
}    

for page in range(1, 4):  # use `while True` to get all pages
    print(f'--- page: {page} ---')
    
    response = requests.get(base, params=params)
    
    #print('status_code:', response.status_code)
    #print('response.ok:', response.ok)
    
    if not response.ok:
        print('response.text:', response.text)
        break
    
    data = response.json()
    #print(data.keys())

    data_responses = data['StudyFieldsResponse']
    #print(data_responses.keys())
    #     dict_keys(['APIVrs', 'DataVrs', 'Expression', 'NStudiesAvail', 'NStudiesFound', 'MinRank', 'MaxRank', 'NStudiesReturned', 'FieldList', 'StudyFields'])
    #print('APIVrs :',          data_responses['APIVrs'])
    #print('DataVrs:',          data_responses['DataVrs'])
    #print('Expression:',       data_responses['Expression'])
    #print('NStudiesAvail:',    data_responses['NStudiesAvail'])
    #print('NStudiesFound:',    data_responses['NStudiesFound'])
    #print('MinRank:',          data_responses['MinRank'])
    #print('MaxRank:',          data_responses['MaxRank'])
    #print('NStudiesReturned:', data_responses['NStudiesReturned'])
    #print('FieldList:',        data_responses['FieldList'])
    
    # I use `[:10]` to reduce results on screen
    for index, item in enumerate(data_responses['StudyFields'][:10], 1):
        print(f'{page:2},{index:4}:', item['BriefTitle'])
    
    # next page
    params['min_rnk'] += 1000
    params['max_rnk'] += 1000

Результат:

--- page: 1 ---
 1,   1: ['Diabetes Pueblo Program - Application and Acceptability of Culturally Appropriate Latino Education for Insulin Therapy']
 1,   2: ['Best Practice Study of Diabetes Type 2 Management in Primary Care in Switzerland']
 1,   3: ['Diabetes and Partnership: Evaluation of a Diabetes Education Module']
 1,   4: ['Diabetes and Sports: Evaluation of a Diabetes Education Module']
 1,   5: ['Diabetes and Travel: Evaluation of a Diabetes Education Module - a Randomized Controlled Trial (PRIMO_Travel)']
 1,   6: ['Diabetes and Social Issues: Evaluation of a Diabetes Education Module']
 1,   7: ['Evaluation of a Diabetes-specific Cognitive Behavioural Treatment for Subthreshold Depression']
 1,   8: ['Evaluating the Influence of Diabetes Stigma on Medication Adherence']
 1,   9: ['Diabetes to Go: Inpatient Education Implementation']
 1,  10: ['My Diabetes Care Mobile: A Usability Study']
--- page: 2 ---
 2,   1: ['Left Ventricular DYsfunction in DiAbetes']
 2,   2: ['Glucose Variability in Patients With Type 1 Diabetes With a Sucrose-added Diet']
 2,   3: ['Surgery Versus Best Medical Management for the Long Term Remission of Type 2 Diabetes and Related Diseases (REMISSION)']
 2,   4: ['Evaluation of an Intervention for Young Adults With Diabetes: Resilient, Empowered, Active Living-Telehealth (REAL-T)']
 2,   5: ['Telehealth for Diabetes Self-Management']
 2,   6: ['Comparison of Visceral Adipokines Visfatin, Vaspin and Omentin Levels in Gestational Diabetes Mellitus Pregnant Women']
 2,   7: ["Uncovering the 'ORIGINS' of Diabetes"]
 2,   8: ['Using of Ice Cream for Diagnosis of Diabetes Mellitus']
 2,   9: ['Validation Study of a Peri-intervention Diabetes Management Algorithm (DIAPI) for Endoscopy Procedures']
 2,  10: ['Vaccination Coverage in People Linving With Diabetes']
--- page: 3 ---
 3,   1: ['A Study of People With Type 2 Diabetes Who Have Diseases That Affect the Heart and Blood Vessels. The Study Will Also Look at the Treatment of a Group of These People in Hospitals in Portugal']
 3,   2: ['Randomized, Double-blind, Active-controlled, Study of Rivoglitazone in Type 2 Diabetes Mellitus']
 3,   3: ['Establishing Ambulatory Glucose Profiles for People Without Diabetes Using CGM Data']
 3,   4: ['A Pragmatic Approach to Lower Diabetes Risk After Gestational Diabetes']
 3,   5: ['Effect of GLP-1 on Insulin-dose, Risk of Hypoglycemia and Gastric Emptying Rate in Patients With Type 1 Diabetes']
 3,   6: ['The Influence of Glycemic Control and Obesity on Energy Balance and Metabolic Flexibility in Type 1 Diabetes']
 3,   7: ['Efficacy of the Digital Platform for Diabetes Care Compared to Usual Care in Patients Diagnosed With Type 2 Diabetes.']
 3,   8: ['Effect of Short-Term Beta-Cell Rest in Adolescents and Young Adults With Type 2 Diabetes Mellitus']
 3,   9: ['Weight Loss Study for People With Type 2 Diabetes']
 3,  10: ['Oral Glucose Tolerance Testing After Gestational Diabetes']

Обновлено:

Версия для new API:

Есть документ Руководство по миграции API | ClinicalTrials.gov, где объясняются изменения.

Если вы получили JSON, вам придется использовать поля, как в old API, т.е. NCTId. Поля типа NCT Number должны работать только тогда, когда вы получаете CSV, но я не пытался получить CSV, чтобы проверить это.

JSON есть поле nextPageToken, которое вы должны использовать как params['pageToken'], чтобы перейти на следующую страницу.

import requests

base = 'https://clinicaltrials.gov/api/v2/studies'  # new API

# fields for CSV and JSON: https://clinicaltrials.gov/data-api/about-api/csv-download

# new API fields - for CSV
extract_fields = [
    'NCT Number',
    'Status',
    'Conditions',
    'Interventions',
    'Sponsor',
    'Study Type',
    'Collaborators',
    'Acronym',
    'Outcome Measures',
    'Sex',
    'Age',
    'Phase',
    'Enrollment',
    'Funder type',
    'Study Design',
    'Other IDs',
    'Study Start',
    'Primary Completion',
    'Study Completion',
    'First Posted',
    'Results First Posted',
    'Last Update Posted',
    'Locations',
    'Study Documents'
]

# new API fields - for JSON
extract_fields = [
    'NCTId',
    'Condition',
    'BriefTitle',
]

# new API fields
params = {
    'fields': ",".join(extract_fields), 
    'query.term': 'Diabetes',
    'pageSize': 1000,
    'format': 'json', 
    #'pageToken': None  # first page doesn't need it
}    

for page in range(1, 4):  # use `while True` to get all pages
    print(f'--- page: {page} ---')
    
    response = requests.get(base, params=params)
    
    #print('status_code:', response.status_code)
    #print('response.ok:', response.ok)
    
    if not response.ok:
        print('response.text:', response.text)
        break
    
    data = response.json()
    #print(data.keys())
    data_responses = data['studies']
    #print(data_responses)
    
    # I use `[:10]` to reduce results on screen
    for index, item in enumerate(data_responses[:10], 1):
        #print(item)
        item = item['protocolSection']['identificationModule']
        #print(item)
        print(f'{page:2},{index:4}:', item['briefTitle'])
    
    # next page
    #next_page_token = response.headers.get('x-next-page')  # (probably) for `CSV`
    next_page_token = data['nextPageToken']                 # for `JSON`
    #print('x-next-page', next_page_token)    
    params['pageToken'] = next_page_token

Я пытаюсь вызвать с помощью v2 API вместо API v1.

URL-адрес

GET https://clinicaltrials.gov/api/v2/studies?query.titles=Diabetes&pageSize = {size}&pageToken = {nextPageToken}

NextPageToken — это механизм, используемый в API для разбиения на страницы, который помогает поэтапно получать большие наборы данных. Когда вы делаете первоначальный запрос к API, поддерживающему разбиение на страницы, ответ часто включает в себя nextPageToken. Затем этот токен можно использовать в качестве параметра в последующем запросе API для получения следующего набора данных. Если ответ содержит nextPageToken, вы продолжаете отправлять запросы с новым токеном; если токен отсутствует, это означает, что больше нет страниц для извлечения, и вы можете завершить цикл. Этот метод эффективен для управления большими наборами данных за счет разбиения их на управляемые фрагменты, снижения нагрузки на сервер и сокращения времени отклика.

Во многих ответах API, особенно в тех, которые возвращают сложные структуры данных, не все поля или ключи могут присутствовать в каждом элементе. Это может произойти по нескольким причинам, например, из-за различий в данных, доступных для разных записей, или изменений в модели данных с течением времени.

Чтобы надежно обрабатывать такую ​​изменчивость в Python, вы обычно используете метод .get(), предоставляемый словарями. Этот метод позволяет вам попытаться получить значение из словаря, используя указанный ключ, а также позволяет указать значение по умолчанию, если ключ не найден. Вот как это работает в контексте вашего примера:

acronym = study['protocolSection']['identificationModule'].get('acronym', 'Unknown')

Доступ к словарю: Study['protocolSection']['identificationModule'] обращается к вложенному словарю по ключам 'protocolSection' и 'identificationModule'. Использование .get() для безопасного поиска. Метод .get() используется для безопасной попытки получить значение, связанное с ключевой «аббревиатурой».

Один пример ответа JSON

{
    "studies": [
        {
            "protocolSection": {
                "identificationModule": {
                    "nctId": "NCT00435240",
                    "orgStudyIdInfo": {
                        "id": "H4 726/2004"
                    },
                    "organization": {
                        "fullName": "Lund University Hospital",
                        "class": "OTHER"
                    },
                    "briefTitle": "Paleolithic Diet in the Treatment of Diabetes Type 2 in Primary Health Care"
                },
                "statusModule": {
                    "statusVerifiedDate": "2008-05",
                    "overallStatus": "COMPLETED",
                    "expandedAccessInfo": {
                        "hasExpandedAccess": false
                    },
                    "startDateStruct": {
                        "date": "2005-01"
                    },
                    "primaryCompletionDateStruct": {
                        "date": "2007-09",
                        "type": "ACTUAL"
                    },
                    "completionDateStruct": {
                        "date": "2007-09",
                        "type": "ACTUAL"
                    },
                    "studyFirstSubmitDate": "2007-02-13",
                    "studyFirstSubmitQcDate": "2007-02-13",
                    "studyFirstPostDateStruct": {
                        "date": "2007-02-14",
                        "type": "ESTIMATED"
                    },
                    "lastUpdateSubmitDate": "2008-05-13",
                    "lastUpdatePostDateStruct": {
                        "date": "2008-05-15",
                        "type": "ESTIMATED"
                    }
                },
                "sponsorCollaboratorsModule": {
                    "responsibleParty": {
                        "oldNameTitle": "Dr Tommy Jönsson",
                        "oldOrganization": "Department of Clinical Sciences, Medical Faculty, Lund University Hospital, Sweden"
                    },
                    "leadSponsor": {
                        "name": "Lund University Hospital",
                        "class": "OTHER"
                    }
                },
                "descriptionModule": {
                    "briefSummary": "There is uncertainty about the optimal diet in the prevention and treatment of diabetes type 2. Earlier studies have generally focused on intakes of fat, protein, carbohydrate, fiber, fruit and vegetables. This study is based on another approach which compares foods that were available during human evolution with more recently introduced ones. The basic tenet from evolutionary biology is that if human physiology is less adapted to a relatively recently introduced diet based on agriculture, this could cause disturbances to human physiology and ultimately lead to diseases. Epidemiological studies indicates that diabetes mellitus type 2 is absent or near absent in populations eating a Palaeolithic (\"Old Stone Age\") diet which is free from food items produced in agriculture or the food industry. Our study hypothesis is that a Palaeolithic diet is better than the standard diabetes diet recommended today in treating diabetes type 2.\n\nFifteen patients with diabetes type 2 have been randomized to\n\n1. a Palaeolithic diet based on lean meat, fish, fruit, vegetables, root vegetables, eggs, and nuts\n2. a standard diabetes diet as recommended by national health authorities.\n\nThe patients eat the diet they have been randomized to for three months and then switches to the other diet for another three months. The study is conducted in Primary Health Care stations."
                },
                "conditionsModule": {
                    "conditions": [
                        "Diabetes Mellitus, Type 2"
                    ]
                },
                "designModule": {
                    "studyType": "INTERVENTIONAL",
                    "phases": [
                        "PHASE2"
                    ],
                    "designInfo": {
                        "allocation": "RANDOMIZED",
                        "interventionModel": "CROSSOVER",
                        "primaryPurpose": "TREATMENT",
                        "maskingInfo": {
                            "masking": "NONE"
                        }
                    },
                    "enrollmentInfo": {
                        "count": 13,
                        "type": "ACTUAL"
                    }
                },
                "armsInterventionsModule": {
                    "interventions": [
                        {
                            "type": "BEHAVIORAL",
                            "name": "Paleolithic diet"
                        }
                    ]
                },
                "outcomesModule": {
                    "primaryOutcomes": [
                        {
                            "measure": "area under the curve for glucose (AUC Glucose0-120) at the oral glucose tolerance test at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "area under the curve for insulin (AUC Insulin0-120) at the oral glucose tolerance test at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "HbA1C at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "weight at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "waist circumference at baseline, 3 months and 6 months"
                        }
                    ],
                    "secondaryOutcomes": [
                        {
                            "measure": "satiation measured on visual semi-analogous scale at food intake during 4 days av 6 weeks and 12 weeks"
                        },
                        {
                            "measure": "leptin at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "fasting plasma glucose at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "fasting plasma insulin at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "Systolic and diastolic blod pressure at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "Blood lipids at baseline, 3 months and 6 months"
                        },
                        {
                            "measure": "C-reactive protein at baseline, 3 months and 6 months"
                        }
                    ]
                },
                "eligibilityModule": {
                    "eligibilityCriteria": "Inclusion Criteria:\n\n* Adults with capacity to perform study\n* Diabetes Mellitus Type 2\n* C-peptide \\> 0\n* HbA1C \\>5.5\n* Unchanged diabetes treatment during last 3 months\n* Weight and HbA1C varied less than 5% during last 3 months\n* No acute heart disease during last 6 months\n* Unchanged treatment with betablocker last 6 months\n* Unchanged treatment with thyroid hormone substitution last 6 months\n\nExclusion Criteria:\n\n* Treatment with insulin\n* Chronic treatment with steroids (not inhaled)\n* Treatment with Waran (anticoagulant cumarin type)\n* Creatinin \\> 130 micromol/L\n* Elevated liver enzymes (ALAT,ASAT,ALP or GT \\> 4 X upper reference value)\n* Acute heart disease\n* Changed treatment with betablocker or thyroid hormone substitution",
                    "healthyVolunteers": false,
                    "sex": "ALL",
                    "minimumAge": "18 Years",
                    "stdAges": [
                        "ADULT",
                        "OLDER_ADULT"
                    ]
                },
                "contactsLocationsModule": {
                    "overallOfficials": [
                        {
                            "name": "Tommy Jönsson, MD",
                            "affiliation": "Department of Clinical Sciences, Lund University, Lund Sweden",
                            "role": "PRINCIPAL_INVESTIGATOR"
                        }
                    ],
                    "locations": [
                        {
                            "facility": "Lund University Hospital",
                            "city": "Lund",
                            "state": "Skåne",
                            "zip": "22185",
                            "country": "Sweden",
                            "geoPoint": {
                                "lat": 55.70584,
                                "lon": 13.19321
                            }
                        }
                    ]
                },
                "referencesModule": {
                    "references": [
                        {
                            "pmid": "17081292",
                            "type": "BACKGROUND",
                            "citation": "Jonsson T, Ahren B, Pacini G, Sundler F, Wierup N, Steen S, Sjoberg T, Ugander M, Frostegard J, Goransson L, Lindeberg S. A Paleolithic diet confers higher insulin sensitivity, lower C-reactive protein and lower blood pressure than a cereal-based diet in domestic pigs. Nutr Metab (Lond). 2006 Nov 2;3:39. doi: 10.1186/1743-7075-3-39."
                        },
                        {
                            "pmid": "16336696",
                            "type": "BACKGROUND",
                            "citation": "Jonsson T, Olsson S, Ahren B, Bog-Hansen TC, Dole A, Lindeberg S. Agrarian diet and diseases of affluence--do evolutionary novel dietary lectins cause leptin resistance? BMC Endocr Disord. 2005 Dec 10;5:10. doi: 10.1186/1472-6823-5-10."
                        },
                        {
                            "pmid": "15699220",
                            "type": "BACKGROUND",
                            "citation": "Cordain L, Eaton SB, Sebastian A, Mann N, Lindeberg S, Watkins BA, O'Keefe JH, Brand-Miller J. Origins and evolution of the Western diet: health implications for the 21st century. Am J Clin Nutr. 2005 Feb;81(2):341-54. doi: 10.1093/ajcn.81.2.341."
                        },
                        {
                            "pmid": "27216013",
                            "type": "DERIVED",
                            "citation": "Fontes-Villalba M, Lindeberg S, Granfeldt Y, Knop FK, Memon AA, Carrera-Bastos P, Picazo O, Chanrai M, Sunquist J, Sundquist K, Jonsson T. Palaeolithic diet decreases fasting plasma leptin concentrations more than a diabetes diet in patients with type 2 diabetes: a randomised cross-over trial. Cardiovasc Diabetol. 2016 May 23;15:80. doi: 10.1186/s12933-016-0398-1."
                        },
                        {
                            "pmid": "19604407",
                            "type": "DERIVED",
                            "citation": "Jonsson T, Granfeldt Y, Ahren B, Branell UC, Palsson G, Hansson A, Soderstrom M, Lindeberg S. Beneficial effects of a Paleolithic diet on cardiovascular risk factors in type 2 diabetes: a randomized cross-over pilot study. Cardiovasc Diabetol. 2009 Jul 16;8:35. doi: 10.1186/1475-2840-8-35."
                        }
                    ]
                }
            },
            "derivedSection": {
                "miscInfoModule": {
                    "versionHolder": "2024-05-01"
                },
                "conditionBrowseModule": {
                    "meshes": [
                        {
                            "id": "D000003924",
                            "term": "Diabetes Mellitus, Type 2"
                        }
                    ],
                    "ancestors": [
                        {
                            "id": "D000003920",
                            "term": "Diabetes Mellitus"
                        },
                        {
                            "id": "D000044882",
                            "term": "Glucose Metabolism Disorders"
                        },
                        {
                            "id": "D000008659",
                            "term": "Metabolic Diseases"
                        },
                        {
                            "id": "D000004700",
                            "term": "Endocrine System Diseases"
                        }
                    ],
                    "browseLeaves": [
                        {
                            "id": "M7115",
                            "name": "Diabetes Mellitus",
                            "relevance": "LOW"
                        },
                        {
                            "id": "M7119",
                            "name": "Diabetes Mellitus, Type 2",
                            "asFound": "Diabetes Mellitus, Type 2",
                            "relevance": "HIGH"
                        },
                        {
                            "id": "M11639",
                            "name": "Metabolic Diseases",
                            "relevance": "LOW"
                        },
                        {
                            "id": "M25403",
                            "name": "Glucose Metabolism Disorders",
                            "relevance": "LOW"
                        },
                        {
                            "id": "M7862",
                            "name": "Endocrine System Diseases",
                            "relevance": "LOW"
                        }
                    ],
                    "browseBranches": [
                        {
                            "abbrev": "BC18",
                            "name": "Nutritional and Metabolic Diseases"
                        },
                        {
                            "abbrev": "BC19",
                            "name": "Gland and Hormone Related Diseases"
                        },
                        {
                            "abbrev": "All",
                            "name": "All Conditions"
                        }
                    ]
                }
            },
            "hasResults": false
        },

Сохранить как demo.py

import requests
import pandas as pd

# Initial URL for the first API call
base_url = "https://clinicaltrials.gov/api/v2/studies"
params = {
    "query.titles": "Diabetes",
    "pageSize": 100
}

# Initialize an empty list to store the data
data_list = []

# Loop until there is no nextPageToken
while True:
    # Print the current URL (for debugging purposes)
    print("Fetching data from:", base_url + '?' + '&'.join([f"{k} = {v}" for k, v in params.items()]))
    
    # Send a GET request to the API
    response = requests.get(base_url, params=params)

    # Check if the request was successful
    if response.status_code == 200:
        data = response.json()  # Parse JSON response
        studies = data.get('studies', [])  # Extract the list of studies

        # Loop through each study and extract specific information
        for study in studies:
            # Safely access nested keys
            nctId = study['protocolSection']['identificationModule'].get('nctId', 'Unknown')
            overallStatus = study['protocolSection']['statusModule'].get('overallStatus', 'Unknown')
            startDate = study['protocolSection']['statusModule'].get('startDateStruct', {}).get('date', 'Unknown Date')
            conditions = ', '.join(study['protocolSection']['conditionsModule'].get('conditions', ['No conditions listed']))
            acronym = study['protocolSection']['identificationModule'].get('acronym', 'Unknown')

            # Extract interventions safely
            interventions_list = study['protocolSection'].get('armsInterventionsModule', {}).get('interventions', [])
            interventions = ', '.join([intervention.get('name', 'No intervention name listed') for intervention in interventions_list]) if interventions_list else "No interventions listed"
            
            # Extract locations safely
            locations_list = study['protocolSection'].get('contactsLocationsModule', {}).get('locations', [])
            locations = ', '.join([f"{location.get('city', 'No City')} - {location.get('country', 'No Country')}" for location in locations_list]) if locations_list else "No locations listed"
            
            # Extract dates and phases
            primaryCompletionDate = study['protocolSection']['statusModule'].get('primaryCompletionDateStruct', {}).get('date', 'Unknown Date')
            studyFirstPostDate = study['protocolSection']['statusModule'].get('studyFirstPostDateStruct', {}).get('date', 'Unknown Date')
            lastUpdatePostDate = study['protocolSection']['statusModule'].get('lastUpdatePostDateStruct', {}).get('date', 'Unknown Date')
            studyType = study['protocolSection']['designModule'].get('studyType', 'Unknown')
            phases = ', '.join(study['protocolSection']['designModule'].get('phases', ['Not Available']))

            # Append the data to the list as a dictionary
            data_list.append({
                "NCT ID": nctId,
                "Acronym": acronym,
                "Overall Status": overallStatus,
                "Start Date": startDate,
                "Conditions": conditions,
                "Interventions": interventions,
                "Locations": locations,
                "Primary Completion Date": primaryCompletionDate,
                "Study First Post Date": studyFirstPostDate,
                "Last Update Post Date": lastUpdatePostDate,
                "Study Type": studyType,
                "Phases": phases
            })

        # Check for nextPageToken and update the params or break the loop
        nextPageToken = data.get('nextPageToken')
        if nextPageToken:
            params['pageToken'] = nextPageToken  # Set the pageToken for the next request
        else:
            break  # Exit the loop if no nextPageToken is present
    else:
        print("Failed to fetch data. Status code:", response.status_code)
        break

# Create a DataFrame from the list of dictionaries
df = pd.DataFrame(data_list)

# Print the DataFrame
print(df)

# Optionally, save the DataFrame to a CSV file
df.to_csv("clinical_trials_data_complete.csv", index=False)

Запустить его

python demo.py

Результат

Получено 11973 строки

            NCT ID     Acronym Overall Status  Start Date                                        Conditions  ... Primary Completion Date Study First Post Date Last Update Post Date      Study Type         Phases
0      NCT00435240     Unknown      COMPLETED     2005-01                         Diabetes Mellitus, Type 2  ...                 2007-09            2007-02-14            2008-05-15  INTERVENTIONAL         PHASE2
1      NCT03547440       Bioda        UNKNOWN  2015-03-18                          Type 1 Diabetes Mellitus  ...              2017-06-30            2018-06-06            2018-06-06   OBSERVATIONAL  Not Available
2      NCT01847144  Mastermind      COMPLETED     2013-04                                   Type 2 Diabetes  ...                 2015-02            2013-05-06            2018-06-21  INTERVENTIONAL         PHASE4
3      NCT04149392     Unknown      WITHDRAWN  2019-12-16                                          Diabetes  ...              2020-04-06            2019-11-04            2019-12-10  INTERVENTIONAL             NA
4      NCT01847092     Unknown      COMPLETED     2013-05  Chronic Kidney Disease, Type 2 Diabetes Mellitus  ...                 2015-03            2013-05-06            2020-05-27  INTERVENTIONAL         PHASE2
...            ...         ...            ...         ...                                               ...  ...                     ...                   ...                   ...             ...            ...
11968  NCT03947333     Unknown      COMPLETED  2020-03-09                         Diabetes Mellitus, Type 2  ...              2021-01-11            2019-05-13            2022-03-31  INTERVENTIONAL             NA
11969  NCT00562133     Unknown      COMPLETED     2006-12                                 Diabetes Mellitus  ...            Unknown Date            2007-11-21            2007-11-21  INTERVENTIONAL         PHASE3
11970  NCT02846233     Unknown      COMPLETED     2016-08                          Type 2 Diabetes Mellitus  ...                 2018-12            2016-07-27            2020-11-10  INTERVENTIONAL             NA
11971  NCT03017287     Unknown     TERMINATED     2017-01                    Diabetes Mellitus, Type 1 or 2  ...                 2017-05            2017-01-11            2020-09-23  INTERVENTIONAL             NA
11972  NCT04666987     Unknown      COMPLETED  2020-10-21                         Diabetes Mellitus, Type 2  ...              2022-12-30            2020-12-14            2024-04-03   OBSERVATIONAL  Not Available

[11973 rows x 12 columns]