Analyze Jira Issues using BigQuery and Data Studio

Using Jira API to Get Jira Issues

jira_user = 'mrawesome@abcd.com'
jira_apikey = 'xxx'
jira_server = 'https://abcd.atlassian.net/'
jql = 'project = "ABCD" AND created >= "2019/06/28" AND created < "2019/06/29"'
import os, json, copy
from jira import JIRA
jira_user = 'mrawesome@abcd.com'
jira_apikey = 'xxx'
jira_server = 'https://abcd.atlassian.net/'
jql = 'project = "ABCD" AND created >= "2019/06/28" AND created < "2019/06/29"'
temp_file_name = 'jira_created_2019-06-28.json'
prefix = 'jira_created/created=2019-06-28/'
jira = JIRA({'server': jira_server}, basic_auth=(jira_user, jira_apikey))
with open(temp_file_name, 'w') as fp:
for issue in jira.search_issues(jql,maxResults=max_result):
fp.write(json.dumps(issue.raw))
fp.write("\n")

Oops, Need to Cleanse the Data

Upload to GCS and Load to BigQuery

with DAG(dag_id, schedule_interval=schedule_interval,
default_args=default_args) as dag:
jira_to_gcs = PythonOperator(
task_id='jira_to_gcs',
python_callable=jira_to_gcs,
op_kwargs={'bucket_name': gcs_bucket},
provide_context=True
)
gcs_to_bq = GoogleCloudStorageToBigQueryOperator(
task_id='gcs_to_bq',
bucket=gcs_bucket,
source_objects=[ 'jira_created/created=/*.json' ],
destination_project_dataset_table=destination_table_with_partition,
source_format=source_format,
create_disposition='CREATE_IF_NEEDED',
write_disposition='WRITE_TRUNCATE'
)

Finally, Time to Analyze the Data!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store